PapaTramp/security-alert-center
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: backend ingest API, Docker Compose, Ubuntu install guide

Browse Source 
- FastAPI: POST /api/v1/events, GET /health, JSON Schema validation

- PostgreSQL models, Alembic migration, bootstrap API key

- deploy/docker-compose.yml, .env.example

- docs/install-ubuntu-24.04.md, updated work-plan (agents first)
This commit is contained in:
PTah
2026-05-26 20:21:31 +10:00
parent bf1c1c9fe9
commit 69a232d08a
34 changed files with 1139 additions and 134 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.gitignore
+1
View File
@@ -6,6 +6,7 @@ venv/
.env
.env.*
!.env.example
deploy/.env
# Node / frontend (будущее)
node_modules/
README.md
+5 -2
View File
@@ -12,7 +12,9 @@
## Статус проекта
**Фаза документации.** Реализация кода начнётся после утверждения ТЗ и планов.
**Фаза 1B:** backend (FastAPI ingest), Docker Compose. Параллельно — интеграция `UseSAC` в агентах (фаза 1A).
**Подготовка сервера:** [docs/install-ubuntu-24.04.md](docs/install-ubuntu-24.04.md)
## Документация
@@ -24,7 +26,8 @@
| [docs/agent-integration.md](docs/agent-integration.md) | Интеграция агентов, режим `UseSAC` |
| [docs/work-plan.md](docs/work-plan.md) | План работ (этапы разработки) |
| [docs/roadmap.md](docs/roadmap.md) | Дорожная карта продуктовых фаз |
| [docs/deployment.md](docs/deployment.md) | Развёртывание на Ubuntu 24.04 |
| [docs/install-ubuntu-24.04.md](docs/install-ubuntu-24.04.md) | Установка софта на Ubuntu 24.04 (пошагово) |
| [docs/deployment.md](docs/deployment.md) | Развёртывание и эксплуатация |
| [docs/workspace-three-repos.md](docs/workspace-three-repos.md) | Multi-root workspace для трёх репо |
## Целевая платформа
backend/Dockerfile
+21
View File
@@ -0,0 +1,21 @@
FROM python:3.12-slim-bookworm
WORKDIR /app
RUN apt-get update && apt-get install -y --no-install-recommends libpq5 \
&& rm -rf /var/lib/apt/lists/*
COPY backend/requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY backend/app ./app
COPY backend/alembic.ini .
COPY backend/alembic ./alembic
COPY schemas ./schemas
ENV PYTHONPATH=/app
ENV PYTHONUNBUFFERED=1
ENV EVENT_SCHEMA_PATH=/app/schemas/event-schema-v1.json
EXPOSE 8000
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]
backend/README.md
+33 -8
View File
@@ -1,11 +1,36 @@
# Backend (FastAPI)
# Backend — Security Alert Center
**Статус:** не реализован. См. [docs/work-plan.md](../docs/work-plan.md) фаза 1.
FastAPI + PostgreSQL + Alembic.
Планируется:
## Локальная разработка (без Docker)
- `app/main.py` — точка входа
- `app/api/v1/events.py` — ingest
- `app/models/` — SQLAlchemy
- `app/services/` — правила, уведомления
- `alembic/` — миграции
```bash
cd backend
python3.12 -m venv .venv
source .venv/bin/activate # Windows: .venv\Scripts\activate
pip install -r requirements.txt
export DATABASE_URL=postgresql+psycopg2://sac:sac@localhost:5432/sac
export SAC_BOOTSTRAP_API_KEY=sac_dev_test_key_change_me
alembic upgrade head
uvicorn app.main:app --reload --port 8000
```
## Docker (из корня репозитория)
```bash
cd deploy
cp .env.example .env
# отредактировать .env
docker compose up -d
docker compose run --rm migrate
curl http://127.0.0.1:8000/health
```
## API
| Метод | Путь | Auth |
|-------|------|------|
| GET | `/health` | нет |
| POST | `/api/v1/events` | Bearer API key |
См. [docs/agent-integration.md](../docs/agent-integration.md) и [docs/event-schema-v1.json](../docs/event-schema-v1.json).
backend/alembic.ini
+42
View File
@@ -0,0 +1,42 @@
[alembic]
script_location = alembic
prepend_sys_path = .
version_path_separator = os
sqlalchemy.url = driver://user:pass@localhost/dbname
[post_write_hooks]
[loggers]
keys = root,sqlalchemy,alembic
[handlers]
keys = console
[formatters]
keys = generic
[logger_root]
level = WARN
handlers = console
qualname =
[logger_sqlalchemy]
level = WARN
handlers =
qualname = sqlalchemy.engine
[logger_alembic]
level = INFO
handlers =
qualname = alembic
[handler_console]
class = StreamHandler
args = (sys.stderr,)
level = NOTSET
formatter = generic
[formatter_generic]
format = %(levelname)-5.5s [%(name)s] %(message)s
datefmt = %H:%M:%S
backend/alembic/env.py
+46
View File
@@ -0,0 +1,46 @@
from logging.config import fileConfig
from alembic import context
from sqlalchemy import engine_from_config, pool
from app.config import get_settings
from app.database import Base
from app.models import ApiKey, Event, Host # noqa: F401
config = context.config
if config.config_file_name is not None:
fileConfig(config.config_file_name)
target_metadata = Base.metadata
settings = get_settings()
config.set_main_option("sqlalchemy.url", settings.database_url)
def run_migrations_offline() -> None:
url = config.get_main_option("sqlalchemy.url")
context.configure(
url=url,
target_metadata=target_metadata,
literal_binds=True,
dialect_opts={"paramstyle": "named"},
)
with context.begin_transaction():
context.run_migrations()
def run_migrations_online() -> None:
connectable = engine_from_config(
config.get_section(config.config_ini_section, {}),
prefix="sqlalchemy.",
poolclass=pool.NullPool,
)
with connectable.connect() as connection:
context.configure(connection=connection, target_metadata=target_metadata)
with context.begin_transaction():
context.run_migrations()
if context.is_offline_mode():
run_migrations_offline()
else:
run_migrations_online()
backend/alembic/script.py.mako
+25
View File
@@ -0,0 +1,25 @@
"""${message}
Revision ID: ${up_revision}
Revises: ${down_revision | comma,n}
Create Date: ${create_date}
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
${imports if imports else ""}
revision: str = ${repr(up_revision)}
down_revision: Union[str, None] = ${repr(down_revision)}
branch_labels: Union[str, Sequence[str], None] = ${repr(branch_labels)}
depends_on: Union[str, Sequence[str], None] = ${repr(depends_on)}
def upgrade() -> None:
${upgrades if upgrades else "pass"}
def downgrade() -> None:
${downgrades if downgrades else "pass"}
backend/alembic/versions/001_initial.py
+90
View File
@@ -0,0 +1,90 @@
"""initial hosts events api_keys
Revision ID: 001
Revises:
Create Date: 2026-05-26
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
from sqlalchemy.dialects import postgresql
revision: str = "001"
down_revision: Union[str, None] = None
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
op.create_table(
"hosts",
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
sa.Column("agent_instance_id", sa.String(length=128), nullable=True),
sa.Column("hostname", sa.String(length=255), nullable=False),
sa.Column("display_name", sa.String(length=255), nullable=True),
sa.Column("os_family", sa.String(length=32), nullable=False),
sa.Column("os_version", sa.String(length=128), nullable=True),
sa.Column("product", sa.String(length=64), nullable=False),
sa.Column("product_version", sa.String(length=64), nullable=True),
sa.Column("ipv4", sa.String(length=45), nullable=True),
sa.Column("ipv6", sa.String(length=45), nullable=True),
sa.Column("tags", postgresql.JSONB(astext_type=sa.Text()), nullable=True),
sa.Column("use_sac_mode", sa.String(length=32), nullable=True),
sa.Column("last_seen_at", sa.DateTime(timezone=True), server_default=sa.text("now()"), nullable=False),
sa.Column("created_at", sa.DateTime(timezone=True), server_default=sa.text("now()"), nullable=False),
sa.PrimaryKeyConstraint("id"),
sa.UniqueConstraint("agent_instance_id"),
)
op.create_index("ix_hosts_hostname", "hosts", ["hostname"])
op.create_index("ix_hosts_product", "hosts", ["product"])
op.create_table(
"api_keys",
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
sa.Column("name", sa.String(length=128), nullable=False),
sa.Column("key_prefix", sa.String(length=16), nullable=False),
sa.Column("key_hash", sa.String(length=128), nullable=False),
sa.Column("is_active", sa.Boolean(), nullable=False),
sa.Column("created_at", sa.DateTime(timezone=True), server_default=sa.text("now()"), nullable=False),
sa.PrimaryKeyConstraint("id"),
sa.UniqueConstraint("key_hash"),
)
op.create_index("ix_api_keys_key_prefix", "api_keys", ["key_prefix"])
op.create_table(
"events",
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
sa.Column("event_id", sa.String(length=36), nullable=False),
sa.Column("host_id", sa.Integer(), nullable=False),
sa.Column("occurred_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("received_at", sa.DateTime(timezone=True), server_default=sa.text("now()"), nullable=False),
sa.Column("category", sa.String(length=64), nullable=False),
sa.Column("type", sa.String(length=128), nullable=False),
sa.Column("severity", sa.String(length=16), nullable=False),
sa.Column("title", sa.String(length=256), nullable=False),
sa.Column("summary", sa.Text(), nullable=False),
sa.Column("details", postgresql.JSONB(astext_type=sa.Text()), nullable=True),
sa.Column("raw", postgresql.JSONB(astext_type=sa.Text()), nullable=True),
sa.Column("dedup_key", sa.String(length=512), nullable=True),
sa.Column("correlation_id", sa.String(length=36), nullable=True),
sa.Column("payload", postgresql.JSONB(astext_type=sa.Text()), nullable=False),
sa.ForeignKeyConstraint(["host_id"], ["hosts.id"]),
sa.PrimaryKeyConstraint("id"),
sa.UniqueConstraint("event_id", name="uq_events_event_id"),
)
op.create_index("ix_events_event_id", "events", ["event_id"])
op.create_index("ix_events_host_id", "events", ["host_id"])
op.create_index("ix_events_occurred_at", "events", ["occurred_at"])
op.create_index("ix_events_category", "events", ["category"])
op.create_index("ix_events_type", "events", ["type"])
op.create_index("ix_events_severity", "events", ["severity"])
op.create_index("ix_events_dedup_key", "events", ["dedup_key"])
def downgrade() -> None:
op.drop_table("events")
op.drop_table("api_keys")
op.drop_table("hosts")
backend/app/__init__.py
View File
backend/app/api/__init__.py
View File
backend/app/api/v1/__init__.py
View File
backend/app/api/v1/events.py
+43
View File
@@ -0,0 +1,43 @@
from typing import Any
from fastapi import APIRouter, Body, Depends, HTTPException
from pydantic import BaseModel
from sqlalchemy.orm import Session
from app.auth.api_key import get_api_key_auth
from app.config import get_settings
from app.database import get_db
from app.services.ingest import ingest_event
from app.services.schema_validate import validate_event_payload
router = APIRouter(prefix="/events", tags=["events"])
class IngestResponse(BaseModel):
status: str
event_id: str
created: bool
sac_event_url: str
@router.post("", status_code=202, response_model=IngestResponse)
def post_event(
payload: dict[str, Any] = Body(...),
db: Session = Depends(get_db),
_api_key: str = Depends(get_api_key_auth),
) -> IngestResponse:
errors = validate_event_payload(payload)
if errors:
raise HTTPException(status_code=422, detail={"schema_errors": errors[:20]})
event, created = ingest_event(db, payload)
db.commit()
settings = get_settings()
base = settings.sac_public_url.rstrip("/")
return IngestResponse(
status="accepted",
event_id=event.event_id,
created=created,
sac_event_url=f"{base}/api/v1/events/{event.id}",
)
backend/app/api/v1/health.py
+21
View File
@@ -0,0 +1,21 @@
from fastapi import APIRouter, Depends
from sqlalchemy import text
from sqlalchemy.orm import Session
from app.database import get_db
router = APIRouter(tags=["health"])
@router.get("/health")
def health(db: Session = Depends(get_db)) -> dict:
db_status = "ok"
try:
db.execute(text("SELECT 1"))
except Exception:
db_status = "error"
return {
"status": "ok" if db_status == "ok" else "degraded",
"database": db_status,
"service": "security-alert-center",
}
backend/app/api/v1/router.py
+7
View File
@@ -0,0 +1,7 @@
from fastapi import APIRouter
from app.api.v1 import events, health
api_router = APIRouter()
api_router.include_router(health.router)
api_router.include_router(events.router)
backend/app/auth/api_key.py
+40
View File
@@ -0,0 +1,40 @@
import hashlib
import secrets
from fastapi import Depends, HTTPException, Security
from fastapi.security import HTTPAuthorizationCredentials, HTTPBearer
from sqlalchemy import select
from sqlalchemy.orm import Session
from app.database import get_db
from app.models import ApiKey
security_scheme = HTTPBearer(auto_error=False)
def hash_api_key(raw_key: str) -> str:
return hashlib.sha256(raw_key.encode("utf-8")).hexdigest()
def generate_api_key() -> tuple[str, str, str]:
"""Returns (full_key, prefix, hash)."""
raw = f"sac_{secrets.token_urlsafe(32)}"
prefix = raw[:12]
return raw, prefix, hash_api_key(raw)
def verify_api_key(db: Session, raw_key: str) -> bool:
key_hash = hash_api_key(raw_key)
row = db.scalar(select(ApiKey).where(ApiKey.key_hash == key_hash, ApiKey.is_active.is_(True)))
return row is not None
def get_api_key_auth(
credentials: HTTPAuthorizationCredentials | None = Security(security_scheme),
db: Session = Depends(get_db),
) -> str:
if credentials is None or credentials.scheme.lower() != "bearer":
raise HTTPException(status_code=401, detail="Missing or invalid Authorization header")
if not verify_api_key(db, credentials.credentials):
raise HTTPException(status_code=401, detail="Invalid API key")
return credentials.credentials
backend/app/config.py
+33
View File
@@ -0,0 +1,33 @@
from functools import lru_cache
from pathlib import Path
from pydantic_settings import BaseSettings, SettingsConfigDict
ROOT = Path(__file__).resolve().parents[2]
SCHEMA_PATH = ROOT / "schemas" / "event-schema-v1.json"
class Settings(BaseSettings):
model_config = SettingsConfigDict(
env_file=".env",
env_file_encoding="utf-8",
extra="ignore",
)
database_url: str = "postgresql+psycopg2://sac:sac@localhost:5432/sac"
sac_public_url: str = "http://localhost:8000"
jwt_secret: str = "change-me-in-production"
jwt_algorithm: str = "HS256"
jwt_expire_minutes: int = 60 * 24
sac_bootstrap_api_key: str = ""
sac_admin_username: str = "admin"
sac_admin_password: str = ""
event_schema_path: str = str(SCHEMA_PATH) # override: EVENT_SCHEMA_PATH
cors_origins: str = "*"
@lru_cache
def get_settings() -> Settings:
return Settings()
backend/app/database.py
+22
View File
@@ -0,0 +1,22 @@
from collections.abc import Generator
from sqlalchemy import create_engine
from sqlalchemy.orm import DeclarativeBase, Session, sessionmaker
from app.config import get_settings
settings = get_settings()
engine = create_engine(settings.database_url, pool_pre_ping=True)
SessionLocal = sessionmaker(bind=engine, autocommit=False, autoflush=False)
class Base(DeclarativeBase):
pass
def get_db() -> Generator[Session, None, None]:
db = SessionLocal()
try:
yield db
finally:
db.close()
backend/app/main.py
+66
View File
@@ -0,0 +1,66 @@
from contextlib import asynccontextmanager
from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware
from sqlalchemy import select
from app.api.v1.router import api_router
from app.auth.api_key import hash_api_key
from app.config import get_settings
from app.database import SessionLocal
from app.models import ApiKey
def bootstrap_api_key() -> None:
settings = get_settings()
if not settings.sac_bootstrap_api_key:
return
db = SessionLocal()
try:
key_hash = hash_api_key(settings.sac_bootstrap_api_key)
exists = db.scalar(select(ApiKey).where(ApiKey.key_hash == key_hash))
if exists:
return
prefix = settings.sac_bootstrap_api_key[:12]
db.add(
ApiKey(
name="bootstrap",
key_prefix=prefix,
key_hash=key_hash,
is_active=True,
)
)
db.commit()
finally:
db.close()
@asynccontextmanager
async def lifespan(_app: FastAPI):
bootstrap_api_key()
yield
def create_app() -> FastAPI:
settings = get_settings()
app = FastAPI(
title="Security Alert Center",
version="0.1.0",
lifespan=lifespan,
)
origins = [o.strip() for o in settings.cors_origins.split(",") if o.strip()]
app.add_middleware(
CORSMiddleware,
allow_origins=origins if origins != ["*"] else ["*"],
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
)
from app.api.v1.health import router as health_router
app.include_router(api_router, prefix="/api/v1")
app.include_router(health_router)
return app
app = create_app()
backend/app/models/__init__.py
+5
View File
@@ -0,0 +1,5 @@
from app.models.api_key import ApiKey
from app.models.event import Event
from app.models.host import Host
__all__ = ["ApiKey", "Event", "Host"]
backend/app/models/api_key.py
+17
View File
@@ -0,0 +1,17 @@
from datetime import datetime
from sqlalchemy import Boolean, DateTime, String, func
from sqlalchemy.orm import Mapped, mapped_column
from app.database import Base
class ApiKey(Base):
__tablename__ = "api_keys"
id: Mapped[int] = mapped_column(primary_key=True, autoincrement=True)
name: Mapped[str] = mapped_column(String(128))
key_prefix: Mapped[str] = mapped_column(String(16), index=True)
key_hash: Mapped[str] = mapped_column(String(128), unique=True)
is_active: Mapped[bool] = mapped_column(Boolean, default=True)
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now())
backend/app/models/event.py
+30
View File
@@ -0,0 +1,30 @@
from datetime import datetime
from sqlalchemy import DateTime, ForeignKey, String, Text, UniqueConstraint, func
from sqlalchemy.dialects.postgresql import JSONB
from sqlalchemy.orm import Mapped, mapped_column, relationship
from app.database import Base
class Event(Base):
__tablename__ = "events"
__table_args__ = (UniqueConstraint("event_id", name="uq_events_event_id"),)
id: Mapped[int] = mapped_column(primary_key=True, autoincrement=True)
event_id: Mapped[str] = mapped_column(String(36), index=True)
host_id: Mapped[int] = mapped_column(ForeignKey("hosts.id"), index=True)
occurred_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), index=True)
received_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now())
category: Mapped[str] = mapped_column(String(64), index=True)
type: Mapped[str] = mapped_column(String(128), index=True)
severity: Mapped[str] = mapped_column(String(16), index=True)
title: Mapped[str] = mapped_column(String(256))
summary: Mapped[str] = mapped_column(Text)
details: Mapped[dict | None] = mapped_column(JSONB)
raw: Mapped[dict | None] = mapped_column(JSONB)
dedup_key: Mapped[str | None] = mapped_column(String(512), index=True)
correlation_id: Mapped[str | None] = mapped_column(String(36))
payload: Mapped[dict] = mapped_column(JSONB)
host: Mapped["Host"] = relationship(back_populates="events")
backend/app/models/host.py
+28
View File
@@ -0,0 +1,28 @@
from datetime import datetime
from sqlalchemy import DateTime, String, Text, func
from sqlalchemy.dialects.postgresql import JSONB
from sqlalchemy.orm import Mapped, mapped_column, relationship
from app.database import Base
class Host(Base):
__tablename__ = "hosts"
id: Mapped[int] = mapped_column(primary_key=True, autoincrement=True)
agent_instance_id: Mapped[str | None] = mapped_column(String(128), unique=True, index=True)
hostname: Mapped[str] = mapped_column(String(255), index=True)
display_name: Mapped[str | None] = mapped_column(String(255))
os_family: Mapped[str] = mapped_column(String(32))
os_version: Mapped[str | None] = mapped_column(String(128))
product: Mapped[str] = mapped_column(String(64), index=True)
product_version: Mapped[str | None] = mapped_column(String(64))
ipv4: Mapped[str | None] = mapped_column(String(45))
ipv6: Mapped[str | None] = mapped_column(String(45))
tags: Mapped[list | None] = mapped_column(JSONB, default=list)
use_sac_mode: Mapped[str | None] = mapped_column(String(32))
last_seen_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now())
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now())
events: Mapped[list["Event"]] = relationship(back_populates="host")
backend/app/services/ingest.py
+89
View File
@@ -0,0 +1,89 @@
from datetime import datetime, timezone
from sqlalchemy import select
from sqlalchemy.orm import Session
from app.models import Event, Host
def _parse_dt(value: str) -> datetime:
dt = datetime.fromisoformat(value.replace("Z", "+00:00"))
if dt.tzinfo is None:
dt = dt.replace(tzinfo=timezone.utc)
return dt
def upsert_host(db: Session, payload: dict) -> Host:
source = payload.get("source") or {}
host_data = payload.get("host") or {}
agent_id = source.get("agent_instance_id")
hostname = host_data.get("hostname") or "unknown"
product = source.get("product") or "unknown"
host: Host | None = None
if agent_id:
host = db.scalar(select(Host).where(Host.agent_instance_id == agent_id))
if host is None:
host = db.scalar(
select(Host).where(Host.hostname == hostname, Host.product == product).limit(1)
)
if host is None:
host = Host(
agent_instance_id=agent_id,
hostname=hostname,
display_name=host_data.get("display_name"),
os_family=host_data.get("os_family", "linux"),
os_version=host_data.get("os_version"),
product=product,
product_version=source.get("product_version"),
ipv4=host_data.get("ipv4"),
ipv6=host_data.get("ipv6"),
tags=payload.get("tags") or [],
)
db.add(host)
else:
host.hostname = hostname
host.display_name = host_data.get("display_name") or host.display_name
host.os_version = host_data.get("os_version") or host.os_version
host.product_version = source.get("product_version") or host.product_version
host.ipv4 = host_data.get("ipv4") or host.ipv4
host.ipv6 = host_data.get("ipv6") or host.ipv6
if agent_id and not host.agent_instance_id:
host.agent_instance_id = agent_id
host.last_seen_at = datetime.now(timezone.utc)
db.flush()
return host
def ingest_event(db: Session, payload: dict) -> tuple[Event, bool]:
"""
Returns (event, created).
created=False if event_id already exists (idempotent).
"""
event_id = payload["event_id"]
existing = db.scalar(select(Event).where(Event.event_id == event_id))
if existing:
return existing, False
host = upsert_host(db, payload)
event = Event(
event_id=event_id,
host_id=host.id,
occurred_at=_parse_dt(payload["occurred_at"]),
category=payload["category"],
type=payload["type"],
severity=payload["severity"],
title=payload["title"],
summary=payload["summary"],
details=payload.get("details"),
raw=payload.get("raw"),
dedup_key=payload.get("dedup_key"),
correlation_id=payload.get("correlation_id"),
payload=payload,
)
db.add(event)
db.flush()
return event, True
backend/app/services/schema_validate.py
+24
View File
@@ -0,0 +1,24 @@
import json
from functools import lru_cache
from pathlib import Path
import jsonschema
from jsonschema import Draft202012Validator
from app.config import get_settings
@lru_cache
def get_event_validator() -> Draft202012Validator:
path = Path(get_settings().event_schema_path)
if not path.is_file():
path = Path(__file__).resolve().parents[3] / "schemas" / "event-schema-v1.json"
schema = json.loads(path.read_text(encoding="utf-8"))
Draft202012Validator.check_schema(schema)
return Draft202012Validator(schema)
def validate_event_payload(payload: dict) -> list[str]:
validator = get_event_validator()
errors = sorted(validator.iter_errors(payload), key=lambda e: e.path)
return [e.message for e in errors]
backend/requirements.txt
+16
View File
@@ -0,0 +1,16 @@
fastapi>=0.115.0,<1
uvicorn[standard]>=0.32.0,<1
sqlalchemy>=2.0.36,<3
alembic>=1.14.0,<2
psycopg2-binary>=2.9.10,<3
pydantic>=2.10.0,<3
pydantic-settings>=2.6.0,<3
python-multipart>=0.0.17,<1
jsonschema>=4.23.0,<5
passlib[bcrypt]>=1.7.4,<2
python-jose[cryptography]>=3.3.0,<4
httpx>=0.28.0,<1
# dev
pytest>=8.3.0,<9
pytest-asyncio>=0.24.0,<1
backend/tests/test_health.py
+7
View File
@@ -0,0 +1,7 @@
"""Smoke tests — run with PostgreSQL or skip integration."""
import pytest
def test_placeholder():
assert True
deploy/.env.example
+23
View File
@@ -0,0 +1,23 @@
# Скопировать: cp .env.example .env (или в /etc/security-alert-center/.env)
# PostgreSQL
POSTGRES_USER=sac
POSTGRES_PASSWORD=change-me-long-random
POSTGRES_DB=sac
# API (публичный URL для ссылок в ответах ingest)
SAC_PUBLIC_URL=https://sac.example.com
SAC_API_PORT=8000
# JWT (UI — позже)
JWT_SECRET=change-me-openssl-rand-hex-32
# Первый API key для агентов (заголовок: Authorization: Bearer <ключ>)
# Сгенерировать: python -c "import secrets; print('sac_'+secrets.token_urlsafe(32))"
SAC_BOOTSTRAP_API_KEY=sac_replace_with_generated_key
# Admin UI (фаза 1C)
SAC_ADMIN_USERNAME=admin
SAC_ADMIN_PASSWORD=
CORS_ORIGINS=*
deploy/README.md
+7 -7
View File
@@ -1,10 +1,10 @@
# Deploy
**Статус:** не реализован. См. [docs/deployment.md](../docs/deployment.md).
```bash
cp .env.example .env
nano .env
docker compose up -d --build
docker compose run --rm migrate
```
Планируется:
- `docker-compose.yml`
- `nginx/sacc.conf`
- `.env.example`
- `systemd/` unit-файлы (вариант B)
Полная инструкция для Ubuntu 24.04: [../docs/install-ubuntu-24.04.md](../docs/install-ubuntu-24.04.md)
deploy/docker-compose.yml
+51
View File
@@ -0,0 +1,51 @@
services:
postgres:
image: postgres:16-bookworm
restart: unless-stopped
environment:
POSTGRES_USER: ${POSTGRES_USER:-sac}
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:?set POSTGRES_PASSWORD}
POSTGRES_DB: ${POSTGRES_DB:-sac}
volumes:
- sac_pgdata:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER:-sac} -d ${POSTGRES_DB:-sac}"]
interval: 5s
timeout: 5s
retries: 10
api:
build:
context: ..
dockerfile: backend/Dockerfile
restart: unless-stopped
env_file:
- .env
environment:
DATABASE_URL: postgresql+psycopg2://${POSTGRES_USER:-sac}:${POSTGRES_PASSWORD}@postgres:5432/${POSTGRES_DB:-sac}
SAC_PUBLIC_URL: ${SAC_PUBLIC_URL:-http://localhost:8000}
JWT_SECRET: ${JWT_SECRET:?set JWT_SECRET}
SAC_BOOTSTRAP_API_KEY: ${SAC_BOOTSTRAP_API_KEY:?set SAC_BOOTSTRAP_API_KEY}
EVENT_SCHEMA_PATH: /app/schemas/event-schema-v1.json
CORS_ORIGINS: ${CORS_ORIGINS:-*}
ports:
- "${SAC_API_PORT:-8000}:8000"
depends_on:
postgres:
condition: service_healthy
migrate:
build:
context: ..
dockerfile: backend/Dockerfile
env_file:
- .env
environment:
DATABASE_URL: postgresql+psycopg2://${POSTGRES_USER:-sac}:${POSTGRES_PASSWORD}@postgres:5432/${POSTGRES_DB:-sac}
command: alembic upgrade head
depends_on:
postgres:
condition: service_healthy
volumes:
sac_pgdata:
docs/INDEX.md
+1
View File
@@ -9,6 +9,7 @@
| [event-schema-v1.json](event-schema-v1.json) | JSON Schema событий |
| [work-plan.md](work-plan.md) | План работ по фазам |
| [roadmap.md](roadmap.md) | Дорожная карта версий |
| [install-ubuntu-24.04.md](install-ubuntu-24.04.md) | **Подготовка сервера:** Docker, git, firewall, первый запуск |
| [deployment.md](deployment.md) | Ubuntu 24.04, backup, TLS |
| [workspace-three-repos.md](workspace-three-repos.md) | Cursor multi-root |
docs/deployment.md
+4 -1
View File
@@ -1,6 +1,9 @@
# Развёртывание на Ubuntu 24.04
Руководство для эксплуатации SAC. Реализация `deploy/` — в фазе 1; здесь — целевая архитектура развёртывания.
Руководство для эксплуатации SAC.
> **Подготовка чистого сервера (установка Docker, git, firewall, первый запуск):**
> **[install-ubuntu-24.04.md](install-ubuntu-24.04.md)** — пошаговые команды для администратора.
---
docs/install-ubuntu-24.04.md
+277
View File
@@ -0,0 +1,277 @@
# Подготовка сервера Ubuntu 24.04 для Security Alert Center
Пошаговое руководство: установка софта на **чистый** или обновлённый хост перед развёртыванием SAC.
Целевой способ запуска — **Docker Compose** (рекомендуется). В конце — проверка и ссылка на [deployment.md](deployment.md).
---
## 0. Исходные данные
Запишите до начала:
| Параметр | Пример |
|----------|--------|
| FQDN | `sac.kalinin.example` |
| IP | `10.0.0.50` |
| Пользователь sudo | `deploy` |
| Каталог установки | `/opt/security-alert-center` |
---
## 1. Базовая настройка ОС
```bash
sudo apt update
sudo apt upgrade -y
sudo timedatectl set-timezone Europe/Moscow # или ваша IANA-зона
timedatectl status
```
Опционально — hostname:
```bash
sudo hostnamectl set-hostname sac-01
```
### 1.1. Firewall (ufw)
```bash
sudo apt install -y ufw
sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw allow OpenSSH
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
sudo ufw enable
sudo ufw status verbose
```
SSH оставьте только с доверенных подсетей, если политика безопасности требует:
```bash
# пример: только внутренняя сеть
# sudo ufw allow from 10.0.0.0/8 to any port 22
```
---
## 2. Установка Git
```bash
sudo apt install -y git
git --version
```
Клонирование (после доступа к репозиторию):
```bash
sudo mkdir -p /opt
sudo git clone https://git.kalinamall.ru/PapaTramp/security-alert-center.git /opt/security-alert-center
sudo chown -R "$USER:$USER" /opt/security-alert-center
```
---
## 3. Установка Docker Engine + Compose plugin
Официальный репозиторий Docker для Ubuntu 24.04 (noble):
```bash
sudo apt install -y ca-certificates curl gnupg
sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
sudo chmod a+r /etc/apt/keyrings/docker.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu noble stable" | \
sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt update
sudo apt install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
```
Добавить пользователя в группу `docker` (перелогиниться после):
```bash
sudo usermod -aG docker "$USER"
```
Проверка:
```bash
docker --version
docker compose version
sudo docker run --rm hello-world
```
---
## 4. Каталоги данных и конфигурации
```bash
sudo mkdir -p /etc/security-alert-center
sudo mkdir -p /var/lib/sac/postgres
sudo mkdir -p /var/backups/sac
sudo chown -R "$USER:docker" /var/lib/sac
```
Права на секреты:
```bash
sudo chmod 750 /etc/security-alert-center
```
---
## 5. Конфигурация SAC (.env)
```bash
cd /opt/security-alert-center/deploy
cp .env.example /etc/security-alert-center/.env
chmod 600 /etc/security-alert-center/.env
nano /etc/security-alert-center/.env
```
Обязательно задать (см. комментарии в `.env.example`):
- `POSTGRES_PASSWORD` — длинный случайный пароль
- `JWT_SECRET` — `openssl rand -hex 32`
- `SAC_PUBLIC_URL` — публичный URL, например `https://sac.kalinin.example`
- `SAC_ADMIN_PASSWORD` — пароль первого admin (только первый запуск)
Симлинк для compose (удобно):
```bash
ln -sf /etc/security-alert-center/.env /opt/security-alert-center/deploy/.env
```
---
## 6. Первый запуск SAC
```bash
cd /opt/security-alert-center/deploy
docker compose build
docker compose up -d --build
docker compose run --rm migrate
docker compose ps
```
Проверка health (пока без TLS — с хоста):
```bash
curl -sS http://127.0.0.1:8000/health | jq .
```
Ожидается JSON с `"status": "ok"` и `"database": "ok"`.
Тест ingest (подставьте ключ из `.env` — `SAC_BOOTSTRAP_API_KEY` или созданный в UI):
```bash
curl -sS -X POST http://127.0.0.1:8000/api/v1/events \
-H "Authorization: Bearer ВАШ_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"schema_version": "1.0",
"event_id": "00000000-0000-4000-8000-000000000001",
"occurred_at": "2026-05-26T12:00:00+03:00",
"source": {"product": "ssh-monitor", "product_version": "test"},
"host": {"hostname": "test-host", "os_family": "linux"},
"category": "agent",
"type": "agent.test",
"severity": "info",
"title": "Test",
"summary": "Install check"
}'
```
---
## 7. Nginx и TLS (production)
Установка nginx на хосте **или** использование контейнера из `docker compose` (см. `deploy/docker-compose.yml`).
### 7.1. Certbot (Let's Encrypt), если есть публичный DNS
```bash
sudo apt install -y nginx certbot python3-certbot-nginx
# после настройки server_name в nginx:
sudo certbot --nginx -d sac.kalinin.example
```
### 7.2. Внутренний CA
Используйте корпоративный сертификат в `deploy/nginx/certs/` и смонтируйте в compose.
Агентам (ssh-monitor / Windows) нужно доверять этому CA.
---
## 8. Автозапуск после перезагрузки
Docker Compose с `restart: unless-stopped` в `docker-compose.yml` — сервисы поднимутся сами.
Проверка:
```bash
sudo reboot
# после входа:
docker compose -f /opt/security-alert-center/deploy/docker-compose.yml ps
```
---
## 9. Резервное копирование PostgreSQL
```bash
sudo tee /etc/cron.d/sac-backup <<'EOF'
0 3 * * * root docker compose -f /opt/security-alert-center/deploy/docker-compose.yml exec -T postgres pg_dump -U sac -Fc sac > /var/backups/sac/sac_$(date +\%Y\%m\%d).dump
EOF
```
Ротация старых дампов — `logrotate` или скрипт по политике (14 daily).
---
## 10. Обновление SAC
```bash
cd /opt/security-alert-center
git pull
cd deploy
docker compose build
docker compose up -d
docker compose exec api alembic upgrade head
```
---
## 11. Чеклист готовности сервера
- [ ] Ubuntu 24.04, время и timezone верные
- [ ] ufw: 22 (по политике), 80, 443
- [ ] Docker и compose работают
- [ ] Репозиторий в `/opt/security-alert-center`
- [ ] `/etc/security-alert-center/.env` заполнен, chmod 600
- [ ] `docker compose up` — все контейнеры healthy
- [ ] `GET /health` — ok
- [ ] Тестовый `POST /api/v1/events` — 202
- [ ] TLS настроен (prod)
- [ ] cron backup
---
## 12. Устранение неполадок
| Симптом | Действие |
|---------|----------|
| `database: error` в /health | `docker compose logs postgres api`, проверить `DATABASE_URL` |
| 401 на ingest | неверный API key; см. `SAC_BOOTSTRAP_API_KEY` в .env |
| Нет места на диске | `df -h`, очистка journal/docker: `docker system prune` |
| Порт 8000 занят | изменить mapping в compose или остановить конфликтующий сервис |
---
## См. также
- [deployment.md](deployment.md) — архитектура, порты, эксплуатация
- [agent-integration.md](agent-integration.md) — настройка агентов после поднятия SAC
- [TZ.md](TZ.md) — требования
docs/roadmap.md
+2 -2
View File
@@ -6,9 +6,9 @@
## v0.1 — MVP (ядро SAC)
**Цель:** центр без изменений прод-агентов (тест через curl/mock).
**Цель:** ingest на сервере; агенты в `dual` по мере готовности фазы 1A.
- Ingest API + PostgreSQL
- Ingest API + PostgreSQL (**в работе — фаза 1B**)
- Events, Hosts, базовый UI
- Problems (2 правила)
- Telegram из SAC
docs/work-plan.md
+63 -114
View File
@@ -1,154 +1,103 @@
# План работ — Security Alert Center
План разработки **после утверждения ТЗ**. Код не пишется до завершения фазы 0.
**Актуальный порядок:** сначала подготовка агентов (параллельно с минимальным SAC ingest), затем полный SAC MVP, затем prod rollout `UseSAC=exclusive`.
---
## Фаза 0. Документация и согласование (текущая)
## Фаза 0. Документация ✅
| # | Задача | Статус |
|---|--------|--------|
| 0.1 | ТЗ [TZ.md](TZ.md) | ✅ |
| 0.2 | Архитектура, схема событий, интеграция агентов | ✅ |
| 0.3 | План работ, roadmap, deployment | ✅ |
| 0.4 | Согласование ТЗ с заказчиком | ⏳ |
| 0.5 | Репозиторий на git.kalinamall.ru (создать remote, push) | ⏳ |
| 0.6 | Multi-root workspace для трёх репо | ⏳ |
**Выход:** утверждённое ТЗ v1.0, тег `docs-v1.0` в git.
| 0.1–0.3 | ТЗ, архитектура, схема, deployment | ✅ |
| 0.5 | Репозиторий git.kalinamall.ru | ✅ |
| 0.6 | Multi-root workspace | ✅ |
---
## Фаза 1. MVP — ядро SAC (оценка: 3–4 недели)
## Фаза 1A. Агенты — контракт SAC (параллельно с 1B)
### 1.1. Инфраструктура проекта
**Репозитории:** `ssh-monitor`, `RDP-login-monitor`
**Прод:** везде `UseSAC=off` по умолчанию.
| # | Задача | Зависимости |
|---|--------|-------------|
| 1.1.1 | Scaffold `backend/` FastAPI, структура пакетов | 0.4 |
| 1.1.2 | Scaffold `frontend/` Vue 3 + Vite | 0.4 |
| 1.1.3 | `deploy/docker-compose.yml` (api, postgres, nginx) | 1.1.1 |
| 1.1.4 | Alembic, первая миграция (hosts, events, users) | 1.1.1 |
| 1.1.5 | `.env.example`, документация локального dev | 1.1.3 |
| # | Задача | Статус |
|---|--------|--------|
| 1A.1 | Параметры `UseSAC`, `SAC_URL`, `SAC_API_KEY`, spool | ⏳ |
| 1A.2 | `build_sac_event()` + `send_sac_event()` по schema v1 | ⏳ |
| 1A.3 | `notify_or_sac()`: off / dual / exclusive / fallback | ⏳ |
| 1A.4 | `--check-sac` / `Test-SacConnection` | ⏳ |
| 1A.5 | README агентов | ⏳ |
### 1.2. Ingest API
| # | Задача |
|---|--------|
| 1.2.1 | Модель Event + валидация по JSON Schema |
| 1.2.2 | `POST /api/v1/events`, auth API key |
| 1.2.3 | Идемпотентность по `event_id` |
| 1.2.4 | Авторегистрация host при ingest |
| 1.2.5 | `GET /health` |
| 1.2.6 | Unit-тесты ingest |
### 1.3. UI — базовый
| # | Задача |
|---|--------|
| 1.3.1 | Auth: login, JWT |
| 1.3.2 | Страница Events (таблица, фильтры, пагинация) |
| 1.3.3 | Страница Hosts |
| 1.3.4 | Карточка события |
| 1.3.5 | SSE live-лента (последние N) |
### 1.4. Problems (минимум)
| # | Задача |
|---|--------|
| 1.4.1 | Модель Problem + 2 встроенных правила (brute SSH, missing heartbeat) |
| 1.4.2 | UI Problems: список, ack, resolve |
### 1.5. Уведомления из SAC
| # | Задача |
|---|--------|
| 1.5.1 | Канал Telegram (настройки в env/БД) |
| 1.5.2 | Worker: отправка по правилу severity ≥ warning |
| 1.5.3 | Шаблон сообщения (summary + ссылка на UI) |
### 1.6. Dashboard MVP
| # | Задача |
|---|--------|
| 1.6.1 | API агрегатов: events/hour, failed by type |
| 1.6.2 | UI: 3 виджета (график входов, failed, топ IP) |
### 1.7. Приёмка MVP
| # | Задача |
|---|--------|
| 1.7.1 | Чеклист [TZ.md](TZ.md) §9 |
| 1.7.2 | Развёртывание на тестовом Ubuntu 24.04 |
| 1.7.3 | Демо с curl / mock-agent |
**Выход фазы 1:** тег `v0.1.0-mvp`, работающий SAC без изменений в прод-агентах.
**Выход:** на тестовом хосте `dual` шлёт JSON в SAC + Telegram.
---
## Фаза 2. Интеграция агентов (оценка: 2–3 недели)
## Фаза 1B. SAC — минимальный ingest (текущая разработка)
| # | Задача | Репозиторий |
|---|--------|-------------|
| 2.1 | `UseSAC`, `send_sac_event`, spool | ssh-monitor |
| 2.2 | `UseSAC`, `Send-SacEvent`, spool | RDP-login-monitor |
| 2.3 | `--check-sac`, README | оба |
| 2.4 | E2E: exclusive на 1 Linux + 1 Windows | все три |
| 2.5 | Daily report через SAC | SAC + агенты |
| 2.6 | Режим `fallback` | агенты |
| # | Задача | Статус |
|---|--------|--------|
| 1B.1 | Scaffold backend FastAPI | 🔄 |
| 1B.2 | PostgreSQL + Alembic (hosts, events, api_keys) | 🔄 |
| 1B.3 | `POST /api/v1/events`, `GET /health` | 🔄 |
| 1B.4 | docker-compose + [install-ubuntu-24.04.md](install-ubuntu-24.04.md) | 🔄 |
| 1B.5 | Валидация JSON Schema v1 | ⏳ |
**Выход:** теги `ssh-monitor-x.y`, `rdp-monitor-x.y`, `sac-v0.2.0`.
**Выход:** сервер Ubuntu принимает события от curl и от агентов в `dual`.
---
## Фаза 3. Улучшения эксплуатации (оценка: 2–4 недели)
## Фаза 1C. SAC — MVP UI и оповещения
См. [roadmap.md](roadmap.md): batch ingest, email, правила UI, retention job, экспорт CSV, GeoIP, LDAP.
| # | Задача |
|---|--------|
| 1C.1 | Frontend Vue: Events, Hosts |
| 1C.2 | Auth JWT, admin bootstrap |
| 1C.3 | Problems (базовые правила) |
| 1C.4 | Telegram из SAC |
| 1C.5 | Dashboard (3 виджета), SSE |
**Выход:** тег `v0.1.0-mvp`.
---
## Параллельные потоки
## Фаза 2. Пилот и exclusive
| # | Задача |
|---|--------|
| 2.1 | E2E: 1 Linux + 1 Windows, `UseSAC=exclusive` |
| 2.2 | Daily report / heartbeat через SAC |
| 2.3 | fallback на агентах |
**Выход:** `v0.2.0`.
---
## Фаза 3. Эксплуатация
См. [roadmap.md](roadmap.md).
---
## Gantt (упрощённо)
```mermaid
gantt
title План SAC (упрощённо)
title SAC — актуальный план
dateFormat YYYY-MM-DD
section Документы
Фаза 0 ТЗ :done, f0, 2026-05-26, 3d
Фаза 0 :done, 2026-05-26, 1d
section SAC
MVP backend+UI :f1, after f0, 21d
1B ingest+deploy :active, 2026-05-26, 7d
1C UI+notify :2026-06-02, 14d
section Агенты
ssh-monitor SAC :f2, after f1, 10d
RDP-monitor SAC :f2b, after f1, 10d
1A UseSAC dual :2026-05-26, 10d
section Prod
Пилот exclusive :f3, after f2, 7d
Пилот exclusive :2026-06-16, 7d
```
---
## Роли (рекомендация)
| Роль | Фокус |
|------|--------|
| Backend | API, БД, worker, правила |
| Frontend | UI, графики, SSE |
| DevOps | Ubuntu 24.04, nginx, backup |
| Агенты | ssh-monitor + RDP изменения |
На малой команде — одна person full-stack + выделенное время на агенты.
---
## Definition of Done (общий)
- Код в `main`, проходит lint/test в CI (когда появится).
- Документация обновлена.
- Нет секретов в git.
- Критерии приёмки фазы выполнены.
---
## См. также
- [roadmap.md](roadmap.md) — продуктовые фазы
- [TZ.md](TZ.md) — требования
- [install-ubuntu-24.04.md](install-ubuntu-24.04.md) — подготовка сервера
- [roadmap.md](roadmap.md)
- [TZ.md](TZ.md)