e9d7af3cce
This squashes 13 commits from ma/staging plus a small docstring translation
into a single coherent unit. Three workstreams.
== RBAC v13 redesign ==
- Drops core.viewer/analyst/km_admin/admin hierarchy and the
internal_roles / group_mappings / user_role_grants / plugin_access tables.
- Replaced by user_group_members + resource_grants. Atomic v12→v13 backfill
wrapped in BEGIN/COMMIT; ROLLBACK leaves schema_version at 12 for retry.
- Two authorization primitives in app.auth.access:
require_admin — Admin-group god-mode
require_resource_access(rt, "{path}") — entity-scoped grants
Single DB lookup per request; no session cache; no implies BFS.
- /admin/access UI (single page) replaces /admin/role-mapping +
/admin/plugin-access. CLI `da admin group/grant *` replaces
`da admin role/mapping/grant-role/revoke-role/effective-roles`.
- ResourceType.TABLE listing-only — admins can record table grants,
runtime enforcement still flows through legacy dataset_permissions
(migration plan in docs/TODO-rbac-data-enforcement.md).
== Claude Code marketplace ==
- Aggregated /marketplace.zip + /marketplace.git/* (PAT-gated,
RBAC-filtered, content-addressed cache via dulwich).
- Admin god-mode dropped on the marketplace surface — admins curate
their own view via grants like everyone else.
- Bare-repo cache materializes per RBAC-filtered ETag; stale entries
not pruned in this iteration (disclaimed in git_backend.py docstring).
== #81 #83 #44 security/ops hardening ==
- #81 Group A — orchestrator ATTACH allow-listing (extension/url/alias).
- #81 Group B — Keboola extractor 3-state exit codes:
0 success / 1 total fail / 2 PARTIAL fail
Sync API logs PARTIAL FAILURE alert on exit 2. Operators with binary
alerting must teach it the new partial signal.
- #81 Group C — schema v10 view_ownership; rejects silent overwrite
of a prior connector's view name on collision.
- #81 Group D — extractor-side identifier validation.
- #83 — Jira webhook fail-closed when JIRA_WEBHOOK_SECRET unset
+ path-traversal fix.
- #44 — entire /api/scripts/* surface is admin-only (planted-script +
sandbox-bypass risk closed).
== Web UI polish + deploy fix ==
- /admin/access: live grant-count badges (no stale snapshot revert),
shared-header CSS link added to /catalog and /admin/{tables,permissions},
per-resource-type colored stripes.
- docker-compose.host-mount.yml: bind,rbind so dual-disk hosts don't
silently shadow sub-mounts and write state to the wrong disk.
== OSS vendor-neutralization (waves 1+2) ==
- scripts/grpn/ → scripts/ops/. Customer-specific identifiers
(project IDs, internal hostnames, dev/prod VM IPs, brand names)
replaced with placeholders across code, docs, Terraform, Caddyfile,
OAuth probe, and planning docs. Downstream infra repos that copied
scripts/grpn/agnes-tls-rotate.sh or agnes-auto-upgrade.sh must
update the path.
== Translation ==
- src/repositories/user_groups.py::ensure_system docstring translated
from Czech to English for codebase consistency.
Co-authored-by: Mina Rustamyan <mina@keboola.com>
82 lines
3 KiB
Python
82 lines
3 KiB
Python
"""Seed table_registry with dummy entries across multiple buckets.
|
|
|
|
Used to exercise the /admin/access UI with the new ResourceType.TABLE
|
|
without depending on a real data source. Each entry is registered with
|
|
``is_public=False`` so per-group grants are meaningful (a public table
|
|
would bypass any future enforcement).
|
|
|
|
Idempotent — TableRegistryRepository.register() does an UPSERT via
|
|
ON CONFLICT, so re-running this script just refreshes the rows.
|
|
|
|
Usage:
|
|
python scripts/seed_dummy_tables.py
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
from src.db import get_system_db
|
|
from src.repositories.table_registry import TableRegistryRepository
|
|
|
|
|
|
# (bucket, table_id, name, description)
|
|
DUMMY_TABLES: list[tuple[str, str, str, str]] = [
|
|
# Finance
|
|
("in.c-finance", "in_c_finance_orders_dummy",
|
|
"orders_dummy", "Dummy orders fact table — one row per order."),
|
|
("in.c-finance", "in_c_finance_revenue_daily_dummy",
|
|
"revenue_daily_dummy", "Dummy daily revenue rollup."),
|
|
("in.c-finance", "in_c_finance_customers_dummy",
|
|
"customers_dummy", "Dummy customer dimension."),
|
|
("in.c-finance", "in_c_finance_transactions_dummy",
|
|
"transactions_dummy", "Dummy payment transactions."),
|
|
# Marketing
|
|
("in.c-marketing", "in_c_marketing_campaigns_dummy",
|
|
"campaigns_dummy", "Dummy marketing campaigns metadata."),
|
|
("in.c-marketing", "in_c_marketing_ad_spend_dummy",
|
|
"ad_spend_dummy", "Dummy ad spend by channel and day."),
|
|
("in.c-marketing", "in_c_marketing_channels_dummy",
|
|
"channels_dummy", "Dummy marketing channel dimension."),
|
|
("in.c-marketing", "in_c_marketing_attributions_dummy",
|
|
"attributions_dummy", "Dummy multi-touch attributions."),
|
|
# Product
|
|
("in.c-product", "in_c_product_events_dummy",
|
|
"events_dummy", "Dummy product event stream."),
|
|
("in.c-product", "in_c_product_sessions_dummy",
|
|
"sessions_dummy", "Dummy user session aggregates."),
|
|
("in.c-product", "in_c_product_features_dummy",
|
|
"features_dummy", "Dummy feature flag exposure log."),
|
|
("in.c-product", "in_c_product_releases_dummy",
|
|
"releases_dummy", "Dummy release/deploy timeline."),
|
|
]
|
|
|
|
|
|
def main() -> None:
|
|
conn = get_system_db()
|
|
try:
|
|
repo = TableRegistryRepository(conn)
|
|
before = len(repo.list_all())
|
|
for bucket, table_id, name, description in DUMMY_TABLES:
|
|
repo.register(
|
|
id=table_id,
|
|
name=name,
|
|
source_type="dummy",
|
|
bucket=bucket,
|
|
source_table=name,
|
|
query_mode="local",
|
|
description=description,
|
|
registered_by="seed_dummy_tables",
|
|
is_public=False,
|
|
profile_after_sync=False,
|
|
)
|
|
after = len(repo.list_all())
|
|
bucket_count = len({b for b, _, _, _ in DUMMY_TABLES})
|
|
print(
|
|
f"Seeded {len(DUMMY_TABLES)} tables across {bucket_count} buckets "
|
|
f"(registry: {before} -> {after} rows)."
|
|
)
|
|
finally:
|
|
conn.close()
|
|
|
|
|
|
if __name__ == "__main__":
|
|
main()
|