Отлично! Переходим к финальной стадии — развёртывание онлайн-обучаемой модели в облаке с полным CI/CD, мониторингом и масштабированием. --- # ☁️ #полный #продакшен-#пайплайн: ## FastAPI + Streamlit + Docker + #MLflow + #Evidently + #NGINX + #Docker #Compose --- ## 📦 1. Структура проекта ``` ml-online-pipeline/ │ ├── api/ # FastAPI │ ├── app.py │ └── model_handler.py │ ├── dashboard/ # Streamlit │ └── dashboard.py │ ├── drift_monitor/ # Evidently │ └── monitor.py │ ├── mlruns/ # MLflow (авто) │ ├── models/ # Сохранённые модели │ └── online_model.cbm │ ├── data/ # Симуляция данных │ ├── docker-compose.yml ├── Dockerfile.api ├── Dockerfile.dashboard ├── requirements.txt └── mlflow.yaml # MLflow tracking config ``` --- ## 🐳 2. Docker-образы ### `Dockerfile.api` ```dockerfile FROM python:3.9-slim WORKDIR /app COPY api/ /app COPY requirements.txt . RUN pip install --no-cache-dir -r requirements.txt EXPOSE 8000 CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8000"] ``` ### `Dockerfile.dashboard` ```dockerfile FROM python:3.9-slim WORKDIR /app COPY dashboard/ /app COPY requirements.txt . RUN pip install --no-cache-dir -r requirements.txt EXPOSE 8501 CMD ["streamlit", "run", "dashboard.py", "--server.port=8501", "--server.address=0.0.0.0"] ``` --- ## 🔄 3. `docker-compose.yml` ```yaml version: '3.8' services: api: build: context: . dockerfile: Dockerfile.api ports: - "8000:8000" volumes: - ./models:/app - ./mlruns:/app/mlruns environment: - MODEL_PATH=/app/online_model.cbm depends_on: - mlflow dashboard: build: context: . dockerfile: Dockerfile.dashboard ports: - "8501:8501" volumes: - ./models:/app depends_on: - api mlflow: image: ghcr.io/mlflow/mlflow:2.9.0 command: > mlflow server --backend-store-uri sqlite:///mlruns/mlflow.db --default-artifact-root /mlruns --host 0.0.0.0 --port 5000 ports: - "5000:5000" volumes: - ./mlruns:/mlruns drift-monitor: build: context: . dockerfile: Dockerfile.api # переиспользуем command: python drift_monitor/monitor.py volumes: - ./models:/app - ./data:/app/data depends_on: - api - mlflow nginx: image: nginx:alpine ports: - "80:80" volumes: - ./nginx.conf:/etc/nginx/nginx.conf depends_on: - api - dashboard ``` --- ## 📊 4. Мониторинг дрейфа: `drift_monitor/monitor.py` ```python # monitor.py import time import pandas as pd import numpy as np from evidently import ColumnDriftProfile from evidently.report import Report from evidently.metrics import DataDriftTable import requests import json # Имитация потока reference_data = pd.DataFrame(np.random.randn(1000, 10), columns=[f"feat_{i}" for i in range(10)]) reference_data["target"] = reference_data.sum(axis=1) + np.random.randn(1000) current_data = pd.DataFrame(columns=reference_data.columns) # Отчёт report = Report(metrics=[DataDriftTable()]) while True: try: # Генерируем новую точку new_row = np.random.randn(10).tolist() pred = requests.post("http://api:8000/predict", json={"features": new_row, "target": 0}).json()["prediction"] new_row.append(pred) # Добавляем в буфер current_data.loc[len(current_data)] = new_row if len(current_data) >= 50: report.run(reference_data=reference_data, current_data=current_data) result = report.as_dict() drift_score = result["metrics"][0]["result"]["drift_by_columns"] n_drifted = sum(1 for col in drift_score.values() if col["drift_detected"])