AI-Cognitive-Leap/agnes-the-ai-analyst
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
agnes-the-ai-analyst/app/web/templates/store_upload.html
minasarustamyan 6fe67d5279
Curated marketplace enrichment via agnes-metadata.json + curator metadata (#234) 
* Curated marketplace enrichment via agnes-metadata.json + curator metadata

Adds a second well-defined metadata file `.claude-plugin/agnes-metadata.json`
that upstream marketplace repos can opt into, providing per-plugin (and
per-skill / per-agent) cover photo, demo video URL, doc links, and
category override. The Claude Code marketplace contract is untouched —
agnes-metadata.json + the convention `.agnes/` directory are stripped
from the synthetic Claude Code marketplace served via /marketplace.zip
and /marketplace.git/*, so user instances see a clean Claude Code repo
with no Agnes-only metadata.

Highlights:
- DB schema v32 — adds curator_name + curator_email on marketplace_registry,
  cover_photo_url + video_url + doc_links on marketplace_plugins.
- Mandatory curator at marketplace registration, editable later through
  the admin UI; surfaces on cards + detail pages in place of owner_todo.
- External-asset mirror cache at ${DATA_DIR}/marketplace-cache/<slug>/
  with conditional GET, 60s timeout, 10 MB body cap, SSRF guards, and
  Wikipedia-policy-compliant User-Agent.
- Strict drop semantics — anything Agnes can't deliver as a real PDF /
  Markdown / plain text doc, or a real PNG / JPEG / WebP cover, is
  dropped from the served metadata; UI looks identical to no-entry case
  (gradient placeholder for missing covers, no row in the doc list).
- Doc allowlist + image allowlist enforced on both the curated mirror
  flow and the Flea upload flow (/store/new); shared module
  src/marketplace_assets.py.
- New /api/marketplace/curated/{mp}/{plugin}/{asset,doc,mirrored}/...
  endpoints with path-traversal guards + RBAC + Content-Disposition
  attachment for docs.
- Curator-focused format guide at /marketplace/format-guide; canonical
  source is docs/curated-marketplace-format.md, also linked from the
  admin /admin/marketplaces page next to + Add Marketplace.

See CHANGELOG.md under [Unreleased] for the full breakdown.

* Fix format-guide test assertion to match shortened disclaimer

The 'Flea Market' phrase was trimmed out of the disclaimer in
docs/curated-marketplace-format.md after the curator-focused rewrite.
Update the rendered-HTML test to assert the channel-scoping phrase
that's actually present ('Curated Marketplace channel only') rather
than the 'Flea Market' contrast that's no longer in the doc.

* Drop unused 'version' field from agnes-metadata.json schema

The parser never read it; it was a YAGNI placeholder for future
schema evolution. Curators don't need to wonder what to put there
when adding the file for the first time. Will be re-added if and
when we actually introduce a backwards-incompatible schema change.

* Harden asset mirror against SSRF via redirect + DNS rebinding

The pre-flight _is_safe_url check validated only the initial URL;
urllib.request.urlopen then followed redirects and re-resolved DNS for
the actual connection — both bypassable. Attacker-controlled origin
could 302 to http://169.254.169.254/... and exfil cloud metadata;
attacker-controlled DNS could return public IP first / 127.0.0.1 second.

Replace urlopen call with a shared OpenerDirector wired through three
custom handlers: _SafeRedirectHandler re-runs SSRF allowlist on every
redirect Location (max 5 hops, down from urllib's 10), and
_PinnedHTTPHandler / _PinnedHTTPSHandler connect to the IP that passed
validation rather than re-resolving the hostname. TLS SNI + cert verify
stay bound to the original hostname.

_resolve_safe returns the validated IP (the existing _is_safe_url
2-tuple wrapper stays for backwards compatibility) and rejects round-
robin DNS that mixes a public + private record. _UnsafeRedirectError
is a typed exception so _fetch_url can map redirect blocks to terminal
'rejected' status (not transient 'failed'). _http_open is the single
call site so tests can mock at one well-defined seam.

Tests cover redirect blocking (link-local, loopback), redirect-error
unwrapping inside URLError, pinned-IP connection target, and the
end-to-end DNS-rebinding scenario. Existing tests that mocked
urllib.request.urlopen are migrated to mock _http_open.

* Harden /asset/ endpoint against stored XSS

The endpoint served any file in the cloned marketplace repo with
stdlib-detected Content-Type, so a curator who landed evil.html (or a
renamed evil.png carrying HTML bytes) in the working tree got a
same-origin XSS — the response shares cookie scope with /admin and
/api/me/*.

The asset endpoint is image-only by contract (cover photos referenced
from agnes-metadata.json + inner skill / agent cards), so applying the
same allowlist + magic-bytes pattern that /doc/ already uses closes
the gap without breaking any legitimate use case. Three layered
checks: extension in IMAGE_EXTENSIONS (.png/.jpg/.jpeg/.webp; SVG
excluded — <script> inside SVG executes), validate_image_file magic
bytes (defeats rename-extension attack), Content-Type pinned from the
validated extension (never stdlib mimetypes).

Defense-in-depth: X-Content-Type-Options: nosniff stops browser MIME
sniffing; Content-Security-Policy: default-src 'none' blocks script /
iframe execution even if a future regression let HTML through.

Tests cover the .html extension reject, the renamed-HTML-as-PNG magic-
bytes reject, the .svg reject, and the happy-path PNG with security
headers attached. The pre-existing path-traversal test seeds a real
PNG instead of ok.txt now that the endpoint is image-only.

* Enforce mandatory curator on marketplace PATCH

The POST handler enforced curator_name + curator_email at create time,
but PATCH treated empty / missing curator inputs as 'no change'. Legacy
rows that pre-date v32 (curator_name=NULL) could be edited indefinitely
without ever filling the curator gap, and OWNER_TODO_PLACEHOLDER lingered
on every /marketplace card.

Reject the PATCH with 400 when the post-merge row would persist with
empty curator. The check fires after the existing field-merge logic, so
once-filled rows that don't touch curator still pass through (their
existing values fall through from the DB row). DB column stays nullable
so untouched legacy rows continue to coexist — the gate fires only the
moment an admin opens the edit modal.

Existing PATCH semantics preserved: empty-string input still means 'leave
existing value alone', and once-filled curator can't be cleared (those
test cases pass unchanged). New test seeds a legacy row directly via the
repository, then exercises url-only PATCH (rejected), partial-fill PATCH
(rejected), and full-fill PATCH (succeeds); a follow-up no-curator PATCH
on the now-formed row also passes.

* Drop unused curated-marketplace helpers (PR #234 review)

* build_db_payload — imported by src/marketplace.py but never called.
  The strict-drop semantics it would have implemented were re-written
  inline in _refresh_plugin_cache (see the comment block there). The
  standalone helper still carried the old fall-back-to-original-external-
  URL-on-mirror-failure behaviour, which contradicts the documented
  drop-when-can't-deliver contract — a future contributor who re-wired
  it would have introduced a silent regression. Delete with the helper
  + the import + the comment that referenced it.

* _resolve_marketplace_name — one-line shim with no remaining call
  sites. Callers use _resolve_marketplace_meta which returns name +
  curator together, avoiding the double DB hit the shim exists to
  hide.

* '# noqa: F401  Optional kept for forward-compat' was wrong — Optional
  IS used in src/marketplace.py (line 70 and line 238). Drop the noqa
  comment so a future ruff run doesn't try to remove a real import.

Removing build_db_payload also drops the only remaining use of Optional
in src/marketplace_metadata.py, so the import comes out there too.

* Cap agnes-metadata.json size + catch RecursionError on parse

The reader is invoked once per marketplace per sync and the file is
curator-controlled. Two failure modes were unguarded:

* Multi-GB JSON: path.read_text() pulled the whole file into memory
  before json.loads even ran. A curator with commit access to an
  upstream repo could OOM the sync worker.

* Deeply-nested JSON under any size cap: cpython's recursive object /
  array parser raises RecursionError at ~1000 levels of depth.
  RecursionError is a RuntimeError, not ValueError, so the existing
  catch let it propagate up and abort the entire sync — every other
  marketplace in the same pass got skipped.

Add AGNES_METADATA_MAX_BYTES = 1 MiB (a real metadata file with covers,
docs, categories for ~50 plugins fits in <100 KB so the cap is
generous) and gate the size check on path.stat().st_size before the
body read. Broaden the parse except to (ValueError, RecursionError)
with a unified log line. Both failure modes degrade to the same
empty-dict fall-back the malformed-JSON path already used, so one bad
upstream never aborts the rest of the sync.

Tests cover the size cap firing before json.loads (whitespace-padded
valid JSON exceeding the cap) and the recursion path (5000 nested
arrays — past cpython's default recursion limit but well under the
size cap).

* Persist asset-mirror manifest per body write, before unlink

sync_assets wrote each body atomically (tmp + rename) but persisted
the manifest only at the end of the batch. A kill -9 mid-Phase 2 left
on-disk files the manifest never referenced. Once a curator dropped
that URL from agnes-metadata.json, Phase 3's cleanup had no record of
the file and the orphan stayed forever — there's no GC pass walking
the cache dir today, so disk would slowly bloat.

Phase 2 (body-write iteration): after the in-memory manifest mutation,
persist BEFORE unlinking the previous body. The crash window narrows
from 'all of Phase 2' to 'between persist and unlink' (microseconds).
A persist failure mid-batch keeps the previous body on disk — the on-
disk manifest still references it, and a stale-but-existing file beats
a 404. Cost: one extra tmp+rename per body write; manifest is a few KB
so the overhead is negligible vs. the HTTP fetches.

Phase 3 (curator-removed URLs): same discipline. Collect the to-delete
relpaths, persist the manifest with the entries already gone, THEN
unlink. A crash mid-cleanup leaves at most a microsecond window where
files exist despite the manifest no longer naming them. The next sync
reads the (correct) manifest and the orphan stays orphaned, but the
served state is consistent.

Tests cover per-body persist call count, the post-update on-disk
manifest content, and Phase 3 ordering verified by reading the on-disk
manifest from inside Path.unlink.

* Consolidate marketplace video embeds + format-guide CSS

The YouTube nocookie / Vimeo / <video> / link-fallback detection logic
was duplicated verbatim in marketplace_plugin_detail.html and
marketplace_item_detail.html (~40 JS lines each, with subtly-different
inline styles). Both templates now {% include %} a single
_marketplace_video_embed.html partial inside their IIFE so the regex,
the nocookie attribute set, and the unknown-host link fallback live in
ONE place — future tweaks (new host, new attribute, fixed sandbox flag)
no longer need to be applied twice in lockstep.

The .video-wrap selectors (one inline <style> rule in plugin_detail,
one inline style='...' attribute in item_detail) are replaced by the
existing .video-embed 16:9 wrapper in style-custom.css, with new
.video-embed video / .video-embed a child rules added so the wrapper
handles all four embed shapes uniformly without per-template
positioning.

The 60-line inline <style> block in marketplace_format_guide.html
moves verbatim to style-custom.css under a new 'Marketplace format
guide page' section, scoped to .format-guide so other pages aren't
affected.

No user-visible behaviour change: the rendered HTML for valid
YouTube / Vimeo / mp4 / external links is byte-identical to before,
and the format-guide page renders the same.

* Maintainability cleanup batch (PR #234 review)

#10: drop _path_under from app/api/marketplace.py — it was a byte-
equivalent clone of _safe_join (same Path.resolve(strict=True) +
relative_to() containment check). The three v32 endpoint handlers
(/asset, /doc, /mirrored) now share the existing helper.

#14: rename src/marketplace_assets.py → src/marketplace_asset_validation.py
so the file's purpose is obvious from the name and the previous
overlap with src/marketplace_asset_mirror.py is gone. Six call-site
imports updated in lockstep; CHANGELOG references under [Unreleased]
updated to track the new path.

#11: consolidate the URL builders that resolve
/api/marketplace/curated/<slug>/<plugin>/{asset,doc,mirrored}/...
paths. _internal_asset_url / _internal_doc_url / _mirrored_asset_url
lived in src/marketplace.py, while a copy named _mirrored_url lived
in app/api/marketplace.py with a 'must stay aligned' comment. New
module src/marketplace_urls.py is the single source of truth — both
call sites import from it and a future URL-format tweak only needs
to change one file. The _ROUTE_PREFIX constant collapses the per-
function f-string repetition. The route-handler endpoints themselves
still own the path string literals (keeping the builders identical
to the route declarations remains a checklist item, not a runtime
guarantee).

* Re-key asset-mirror manifest by (plugin, url) + dedup HTTP fetches

The manifest used to be keyed by URL alone, so two plugins in the
same marketplace referencing the same external image (a shared CDN
icon, a common cover) collided on entry.plugin_name — last writer
won. The DB row for the losing plugin then stored a served URL
pointing under the winning plugin's tree, and require_resource_access
denied legitimate access on one side and let the other plugin's user
reach the wrong asset.

In-memory: Dict[Tuple[str, str], MirrorEntry] keyed (plugin_name, url).
On disk: format flips from {url: entry} dict to [entry, ...] list of
self-describing entries (each carries plugin_name + url + the
previous fields). JSON keys can't be tuples; encoding 'plugin::url'
would just shift the parsing burden.

Phase 1 of sync_assets deduplicates fetches by URL — three plugins
sharing one URL share one HTTP request. The conditional-GET prior is
picked from any owning plugin's prior entry; if their etags diverge
(rare) we miss one 304 and pay for a full re-download instead.
Phase 2 still creates a per-(plugin, url) manifest entry pointing
under the plugin's own subdir, and Phase 3 cleanup is keyed the same
way so dropping a URL from one plugin's metadata doesn't disturb
another plugin still referencing it.

Body files stay per plugin (RBAC-clean isolation: deleting plugin A's
cache can't strand plugin B). Bandwidth saved by fetch dedup.

Consumer code re-keyed: src.marketplace._refresh_plugin_cache rebuilt
served_url_for / mirror_status as composite-keyed maps;
app.api.marketplace._resolve_external_via_mirror /
_curated_inner_cover / _curated_inner_enrichment look up by
(plugin_name, url).

Tests cover per-plugin manifest entries with shared URL, the single
HTTP fetch for N plugins, and Phase 3 drop-one-keep-other. All
existing tests migrated to composite key access; v2 list format
assertions verify on-disk shape.

* Migrate asset mirror from urllib.request to httpx

The asset mirror was the only HTTP call site in Agnes still using
urllib.request; every other module (CLI, Jira / OpenMetadata / OpenAI
connectors, scheduler, Telegram bot) already used httpx. The asset
mirror was added in this PR's base commit, so this is the only chance
to bring it into convention before someone copies it as 'the pattern
for HTTP fetches in Agnes'.

Three concrete benefits beyond consistency:

* SSRF defence collapses from five urllib classes
  (_PinnedHTTPConnection, _PinnedHTTPSConnection, _PinnedHTTPHandler,
  _PinnedHTTPSHandler, _SafeRedirectHandler) into one
  _SSRFGuardTransport. httpx invokes handle_request() on every redirect
  hop, so re-validation is free — we don't need a custom redirect
  handler at all.

* DNS-rebinding defence: the transport rewrites request.url.host to the
  SSRF-validated IP before delegating to super().handle_request().
  httpcore connects to whatever URL.host says, so this pins the
  connection without subclassing HTTPSConnection. The original hostname
  goes into the Host header + the sni_hostname extension so TLS / vhost
  routing still bind to the curator-supplied hostname.

* Error handling: one httpx.HTTPError catch-all for transport errors,
  plus specific httpx.TimeoutException / httpx.TooManyRedirects branches
  for clearer diagnostics. Matches the _translate_transport_error shape
  in cli/client.py.

The shared httpx.Client is built lazily at module load (same pattern as
cli/client.py:_get_shared_client) with follow_redirects=True,
max_redirects=5, timeout=HTTP_TIMEOUT_SEC, and our custom transport.

Externally observable behaviour is unchanged: same FetchOutcome
statuses, same manifest format, same conditional GET semantics, same
body-size cap.

Tests migrated from urllib-shaped fakes to httpx-shaped (status_code,
iter_bytes, context manager). Five urllib-specific tests replaced with
httpx equivalents — three transport unit tests + one DNS-rebinding
integration test that verifies host rewrite via monkey-patched
super().handle_request. One test deleted without replacement
(unwrap-URLError-wrapping-an-_UnsafeRedirectError — urllib-specific,
not applicable to httpx).

* Surface curated agnes-metadata enrichment on My Stack tab

GET /api/marketplace/items?tab=my built each curated row from the
on-disk marketplace.json by way of resolve_allowed_plugins, which
doesn't carry the agnes-metadata enrichment columns
(cover_photo_url, video_url, category override, doc_links). The
handler then hard-coded cover_photo_url=None on the synthetic row.
Result: once a user clicked '+ Add to my stack' on a curated card,
the same plugin in tab=my rendered with the gradient placeholder
instead of its cover photo — confusing parity break vs. the curated
tab where the same row goes through MarketplacePluginsRepository
and gets the enriched columns.

Pre-load the enriched marketplace_plugins rows for every marketplace
the user is subscribed to, then look each granted+subscribed plugin
up by (marketplace_id, plugin_name). Fall back to the on-disk
synthetic shape only when the DB row is missing — happens during
the rare race where RBAC is granted before the first sync cycle
ingests the plugin. RBAC gating (granted set from
resolve_allowed_plugins) is unchanged so this fix can't widen
visibility; it just upgrades the data shape behind cards the user
was already going to see.

Per-marketplace list_for_marketplace beats N gets — typical user is
subscribed to <5 marketplaces, so this is at most a handful of
queries vs. one per subscribed plugin.

Regression test seeds a plugin with cover_photo_url + category
override, subscribes the user, hits /api/marketplace/items?tab=my,
and asserts photo_url + category come through. The misleading
'fall through to gradient until the user re-visits the curated tab'
comment is gone.

---------

Co-authored-by: Minas Arustamyan <arustamyan.minas@gmail.com>
2026-05-09 17:01:37 +02:00

748 lines
31 KiB
HTML
Raw Blame History

{% extends "base.html" %}
{% block title %}Upload to Store — {{ config.INSTANCE_NAME }}{% endblock %}
{% block content %}
<style>
/* ── Container override ────────────────────────────────────────── */
/* Width + padding come from .page-shell (style-custom.css) — same
1280px container as /dashboard, /marketplace, /admin/* peers. */
.container:has(.upload-page) > main { margin: 0; padding: 0; }
/* ── Hero (mirrors /setup) ─────────────────────────────────────── */
.upload-hero {
background: linear-gradient(135deg, #0073D1 0%, #0056A3 100%);
border-radius: 12px;
padding: 32px 36px;
margin-bottom: 24px;
box-shadow: 0 4px 16px rgba(0, 115, 209, 0.2);
color: white;
}
.upload-hero .eyebrow {
font-size: 11px; font-weight: 600; text-transform: uppercase;
letter-spacing: 0.8px; color: rgba(255, 255, 255, 0.75);
margin-bottom: 10px;
}
.upload-hero h1 {
font-size: 30px; font-weight: 700; margin: 0 0 8px;
color: #fff; letter-spacing: -0.4px;
}
.upload-hero .sub { font-size: 14px; color: rgba(255,255,255,0.85); line-height: 1.6; }
.upload-hero .meta { display: flex; flex-wrap: wrap; gap: 8px; margin-top: 16px; }
.upload-hero .pill {
background: rgba(255,255,255,0.15); padding: 6px 12px;
border-radius: 6px; font-family: var(--font-mono); font-size: 12px; color: #fff;
}
/* ── Progress bar (4-segment style, matches first-time-setup) ──── */
.progress {
display: flex; gap: 8px; margin-bottom: 20px;
}
.progress .seg {
flex: 1; height: 4px; border-radius: 2px;
background: var(--border, #e5e7eb);
transition: background 0.15s ease;
}
.progress .seg.is-done { background: var(--primary, #0073D1); }
/* ── Card (mirrors /setup) ─────────────────────────────────────── */
.card {
background: var(--surface, #fff);
border: 1px solid var(--border, #e5e7eb);
border-radius: 12px;
box-shadow: 0 1px 2px rgba(0,0,0,0.04);
overflow: hidden;
margin-bottom: 20px;
}
.card-header { padding: 22px 24px 0; display: flex; align-items: center; gap: 12px; }
.card-body { padding: 16px 24px 24px; }
.step-num {
width: 26px; height: 26px;
background: var(--primary, #0073D1); color: #fff;
border-radius: 50%; display: inline-flex; align-items: center;
justify-content: center; font-size: 12px; font-weight: 700; flex-shrink: 0;
}
.card-title { font-size: 16px; font-weight: 600; color: var(--text-primary, #111827); }
.card-sub {
font-size: 13px; color: var(--text-secondary, #6b7280);
line-height: 1.6; margin: 0 0 14px;
}
.card-sub code {
background: var(--border-light, #f3f4f6); padding: 1px 6px;
border-radius: 4px; font-family: var(--font-mono); font-size: 12px;
color: var(--text-primary, #111827);
}
/* ── Form bits ─────────────────────────────────────────────────── */
.field { margin-bottom: 16px; }
.field-label {
display: block; font-size: 13px; font-weight: 500;
color: var(--text-primary, #111827); margin-bottom: 6px;
}
.field-hint { font-size: 12px; color: var(--text-secondary, #6b7280); margin-top: 6px; }
.field input[type=text],
.field input[type=url],
.field textarea,
.field select {
width: 100%; padding: 10px 12px;
border: 1px solid var(--border, #d1d5db); border-radius: 8px;
font-size: 14px; font-family: var(--font-primary, inherit);
box-sizing: border-box; background: #fff;
transition: border-color 0.15s ease, box-shadow 0.15s ease;
}
.field textarea { min-height: 90px; resize: vertical; }
.field input[type=text]:focus,
.field input[type=url]:focus,
.field textarea:focus,
.field select:focus {
outline: none; border-color: var(--primary, #0073D1);
box-shadow: 0 0 0 3px rgba(0, 115, 209, 0.12);
}
/* ── Type tiles (radio cards) ──────────────────────────────────── */
.type-tiles { display: grid; grid-template-columns: repeat(3, 1fr); gap: 10px; }
.type-tiles label {
cursor: pointer; padding: 14px 12px;
border: 1px solid var(--border, #e5e7eb); border-radius: 10px;
text-align: center; transition: all 0.15s ease;
background: #fff;
}
.type-tiles label:hover { border-color: #c7d2fe; }
.type-tiles label.is-active {
border-color: var(--primary, #0073D1);
background: linear-gradient(180deg, rgba(0,115,209,0.06), rgba(0,115,209,0.02));
box-shadow: 0 0 0 3px rgba(0, 115, 209, 0.12);
}
.type-tiles input[type=radio] { display: none; }
.type-tiles .type-name {
font-weight: 600; font-size: 14px; color: var(--text-primary, #111827); margin-bottom: 4px;
}
.type-tiles .type-hint { font-size: 11px; color: var(--text-secondary, #6b7280); line-height: 1.4; }
/* ── ZIP drop zone (file input wrapper) ────────────────────────── */
.file-drop {
border: 1.5px dashed var(--border, #d1d5db);
border-radius: 10px; padding: 18px;
background: var(--background, #f9fafb);
display: flex; align-items: center; gap: 12px;
transition: all 0.15s ease;
}
.file-drop:hover { border-color: var(--primary, #0073D1); background: #fff; }
.file-drop.is-dragover {
border-color: var(--primary, #0073D1); background: #fff;
box-shadow: 0 0 0 4px rgba(0, 115, 209, 0.15);
transform: scale(1.005);
}
.file-drop.has-file { border-style: solid; border-color: #10b981; background: #f0fdf4; }
.file-drop .icon {
width: 36px; height: 36px; border-radius: 8px;
background: rgba(0, 115, 209, 0.1); color: var(--primary, #0073D1);
display: flex; align-items: center; justify-content: center;
flex-shrink: 0; font-size: 16px;
}
.file-drop.has-file .icon { background: rgba(16, 185, 129, 0.1); color: #10b981; }
.file-drop .file-info { flex: 1; min-width: 0; }
.file-drop .file-info .label { font-size: 13px; font-weight: 500; color: var(--text-primary, #111827); }
.file-drop .file-info .label .filename { color: var(--primary, #0073D1); margin-left: 6px; }
.file-drop.has-file .file-info .label { color: #047857; }
.file-drop .file-info .meta { font-size: 12px; color: var(--text-secondary, #6b7280); margin-top: 2px; }
.file-drop input[type=file] { display: none; }
.file-drop button {
appearance: none; padding: 7px 14px;
border: 1px solid var(--border, #d1d5db); background: #fff;
color: var(--text-primary, #111827); border-radius: 8px;
font-size: 12px; font-weight: 500; cursor: pointer;
flex-shrink: 0; font-family: var(--font-primary, inherit);
}
.file-drop button:hover { border-color: var(--primary, #0073D1); color: var(--primary, #0073D1); }
/* ── Doc list ──────────────────────────────────────────────────── */
.doc-list { display: flex; flex-direction: column; gap: 6px; margin-bottom: 8px; }
.doc-item {
display: flex; align-items: center; gap: 10px; justify-content: space-between;
padding: 8px 12px; border: 1px solid var(--border, #e5e7eb);
border-radius: 8px; background: var(--background, #f9fafb); font-size: 13px;
}
.doc-item .name { flex: 1; min-width: 0; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; color: var(--text-primary, #111827); }
.doc-item .size { color: var(--text-secondary, #6b7280); flex-shrink: 0; font-size: 12px; }
.doc-item button {
appearance: none; border: none; background: transparent;
color: #b91c1c; cursor: pointer; font-size: 16px;
padding: 0 4px; line-height: 1;
}
.doc-add {
appearance: none; padding: 8px 14px;
border: 1px dashed var(--border, #d1d5db); background: transparent;
color: var(--text-secondary, #6b7280); border-radius: 8px;
font-size: 13px; cursor: pointer; align-self: flex-start;
font-family: var(--font-primary, inherit);
}
.doc-add:hover { border-color: var(--primary, #0073D1); color: var(--primary, #0073D1); }
/* ── Buttons (mirror /setup primary CTA) ───────────────────────── */
.actions { display: flex; gap: 10px; align-items: center; margin-top: 8px; }
.btn-primary {
display: inline-flex; align-items: center; gap: 8px;
font-family: var(--font-primary, inherit);
font-size: 14px; font-weight: 600; color: #fff;
background: var(--primary, #0073D1); border: none;
border-radius: 8px; padding: 11px 22px; cursor: pointer;
transition: all 0.15s ease;
}
.btn-primary:hover {
background: var(--primary-dark, #0056A3);
transform: translateY(-1px);
box-shadow: 0 4px 14px rgba(0, 115, 209, 0.25);
}
.btn-primary:disabled { opacity: 0.7; cursor: wait; transform: none; box-shadow: none; }
.btn-secondary {
appearance: none; padding: 10px 18px;
border: 1px solid var(--border, #d1d5db); background: #fff;
color: var(--text-secondary, #6b7280); border-radius: 8px;
font-size: 14px; font-weight: 500; cursor: pointer;
font-family: var(--font-primary, inherit);
transition: all 0.15s ease;
}
.btn-secondary:hover { border-color: var(--primary, #0073D1); color: var(--primary, #0073D1); }
.btn-link {
color: var(--text-secondary, #6b7280); text-decoration: none;
font-size: 13px; padding: 10px 8px;
}
.btn-link:hover { color: var(--text-primary, #111827); }
/* ── Status banner ─────────────────────────────────────────────── */
/* `display: flex` on .banner overrides the user-agent default for the
`hidden` attribute, leaving the banner visible even with no error.
Force the override here so the empty banner stays out of the layout. */
.banner[hidden] { display: none !important; }
.banner {
padding: 12px 16px; border-radius: 8px; margin-bottom: 16px;
font-size: 13px; line-height: 1.5; display: flex; align-items: flex-start; gap: 10px;
}
.banner.error { background: #fef2f2; color: #b91c1c; border: 1px solid #fecaca; }
.banner.success { background: #f0fdf4; color: #16a34a; border: 1px solid #bbf7d0; }
.banner .ico { flex-shrink: 0; font-size: 16px; line-height: 1.5; }
/* Preserve newlines in structured upload-error messages (multi-line
finding lists). textContent assignment wouldn't render them
otherwise. */
.banner > span { white-space: pre-wrap; }
/* ── Step swap ─────────────────────────────────────────────────── */
.step { display: none; }
.step.is-active { display: block; }
</style>
<div class="upload-page page-shell">
<div class="upload-hero">
<div class="eyebrow">Store</div>
<h1>Upload an entity</h1>
<p class="sub">
Share a skill, agent, or plugin with everyone on this instance.
Display name will be suffixed with
<code style="background: rgba(255,255,255,0.18); padding: 1px 6px; border-radius: 4px; font-family: var(--font-mono); font-size: 12px;">-by-{{ session.user.email.split('@')[0] }}</code>
so it doesn't collide.
</p>
</div>
<div class="progress">
<div class="seg is-done" id="seg-1"></div>
<div class="seg" id="seg-2"></div>
</div>
<div id="banner" class="banner error" hidden>
<span class="ico">!</span><span id="banner-text"></span>
</div>
<!-- ─── Step 1: Type + ZIP ──────────────────────────────────────── -->
<div id="step-1" class="step is-active">
<div class="card">
<div class="card-header">
<div class="step-num">1</div>
<div class="card-title">Type &amp; ZIP</div>
</div>
<div class="card-body">
<p class="card-sub">Pick what you're uploading and the ZIP archive.
The server validates the layout when you click <code>Next</code>.</p>
<div class="field">
<label class="field-label">Type</label>
<div class="type-tiles" id="type-tiles">
<label class="is-active">
<input type="radio" name="type" value="skill" checked>
<div class="type-name">Skill</div>
<div class="type-hint">Folder with SKILL.md</div>
</label>
<label>
<input type="radio" name="type" value="agent">
<div class="type-name">Agent</div>
<div class="type-hint">.md file with name + description frontmatter</div>
</label>
<label>
<input type="radio" name="type" value="plugin">
<div class="type-name">Plugin</div>
<div class="type-hint">Directory with .claude-plugin/plugin.json</div>
</label>
</div>
</div>
<div class="field">
<label class="field-label">ZIP archive</label>
{# Drop zone uses a <div> rather than <label> so the implicit
label→input click pickup doesn't fire on top of our explicit
JS handler — that combination opened the file picker twice. #}
<div class="file-drop" id="zip-drop">
<div class="icon">📦</div>
<div class="file-info">
<div class="label" id="zip-label">No file selected</div>
<div class="meta" id="zip-meta">Drag & drop a .zip here, or click Choose file</div>
</div>
<input type="file" id="zip" accept=".zip">
<button type="button" id="zip-pick">Choose file</button>
</div>
</div>
<div class="actions">
<button type="button" class="btn-primary" id="next-btn">Next →</button>
<a href="/store" class="btn-link">Cancel</a>
</div>
</div>
</div>
</div>
<!-- ─── Step 2: Details ─────────────────────────────────────────── -->
<div id="step-2" class="step">
<div class="card">
<div class="card-header">
<div class="step-num">2</div>
<div class="card-title">Details</div>
</div>
<div class="card-body">
<p class="card-sub">Pre-filled from the ZIP's frontmatter — change anything you want.
Name and description are required; everything below is optional.</p>
<div class="field">
<label class="field-label" for="name">Name</label>
<input type="text" id="name" required placeholder="my-awesome-skill">
<div class="field-hint">Lowercase letters, digits, hyphens. Max 64 characters.</div>
</div>
<div class="field">
<label class="field-label" for="description">Description</label>
<textarea id="description" placeholder="What does it do?"></textarea>
</div>
<div class="field">
<label class="field-label" for="category">Category</label>
<select id="category">
<option value="">— None —</option>
{% for cat in categories %}
<option value="{{ cat }}">{{ cat }}</option>
{% endfor %}
</select>
<div class="field-hint">Subject area of your entity — helps people filter the Store.</div>
</div>
<div class="field">
<label class="field-label">Cover photo <span style="color:var(--text-secondary,#6b7280);font-weight:400;">(optional)</span></label>
<div class="file-drop" id="photo-drop">
<div class="icon">🖼</div>
<div class="file-info">
<div class="label" id="photo-label">No photo</div>
<div class="meta">Drag & drop or click Choose · PNG / JPEG / WebP only · Max 5 MB</div>
</div>
<input type="file" id="photo" accept="image/jpeg,image/png,image/webp">
<button type="button" id="photo-pick">Choose</button>
</div>
</div>
<div class="field">
<label class="field-label" for="video_url">Video URL <span style="color:var(--text-secondary,#6b7280);font-weight:400;">(optional)</span></label>
<input type="url" id="video_url" placeholder="https://...">
</div>
<div class="field">
<label class="field-label">Documentation <span style="color:var(--text-secondary,#6b7280);font-weight:400;">(optional)</span></label>
<div class="doc-list" id="doc-list"></div>
<input type="file" id="doc-input" hidden
accept=".pdf,.md,.markdown,.txt,application/pdf,text/markdown,text/plain">
<button type="button" class="doc-add" id="add-doc-btn">+ Add file</button>
<div class="field-hint">PDF, Markdown (.md/.markdown), or plain text (.txt). Max 10 MB per file.</div>
</div>
<div class="actions">
<button type="button" class="btn-primary" id="finish-btn">Finish</button>
<button type="button" class="btn-secondary" id="back-btn">← Back</button>
</div>
</div>
</div>
</div>
</div>
<script>
const banner = document.getElementById('banner');
const bannerText = document.getElementById('banner-text');
const nextBtn = document.getElementById('next-btn');
const finishBtn = document.getElementById('finish-btn');
const backBtn = document.getElementById('back-btn');
const zipInput = document.getElementById('zip');
const zipPick = document.getElementById('zip-pick');
const zipDrop = document.getElementById('zip-drop');
const zipLabel = document.getElementById('zip-label');
const zipMeta = document.getElementById('zip-meta');
const photoInput = document.getElementById('photo');
const photoPick = document.getElementById('photo-pick');
const photoDrop = document.getElementById('photo-drop');
const photoLabel = document.getElementById('photo-label');
const addDocBtn = document.getElementById('add-doc-btn');
const docInput = document.getElementById('doc-input');
const docList = document.getElementById('doc-list');
let zipFile = null;
let docs = [];
// Server returns short machine codes in `detail`. Map them to human-friendly
// sentences here. Any unknown code falls back to a generic prefix + the raw
// code so debugging stays possible without leaking jargon.
const ERROR_MESSAGES = {
// Step 1 — validation
invalid_type: 'Pick one of: skill, agent, or plugin.',
zip_invalid: 'That file isn\'t a valid ZIP archive.',
zip_unsafe_path: 'The ZIP contains a path that escapes the archive — refuse to extract.',
zip_missing_skill_md: 'A Skill ZIP must contain a SKILL.md file.',
zip_missing_agent_md_with_frontmatter:
'An Agent ZIP must contain a Markdown file with name + description in YAML frontmatter.',
zip_missing_claude_plugin_json:
'A Plugin ZIP must contain a .claude-plugin/plugin.json file at its root.',
plugin_json_invalid: 'The .claude-plugin/plugin.json file is not valid JSON.',
zip_looks_like_skill:
'This ZIP contains a SKILL.md — looks like a Skill, not an Agent. Switch the type to Skill or remove SKILL.md.',
zip_looks_like_plugin:
'This ZIP contains a .claude-plugin/plugin.json — looks like a Plugin, not the type you picked. Switch the type to Plugin.',
// Step 1 + 2 — file size
file_too_large: 'File too large — max 50 MB for the ZIP, 5 MB for photos, 10 MB per doc.',
// Step 2 — metadata
missing_name:
'Name is required. Either fill it in here or add a `name:` field to your frontmatter.',
invalid_name_format:
'Name must be lowercase letters, digits, and hyphens only (max 64 characters).',
invalid_category: 'Pick a category from your own groups, or leave it blank.',
invalid_email:
'Couldn\'t derive a username from your email. Contact your administrator.',
conflict_owner_name:
'You already have a Store entity with this name. Each owner needs unique names — pick a different one or delete the existing entity first.',
// Step 2 — photo
photo_unsupported_format: 'Photo must be JPG, PNG, or WebP.',
// Other
entity_not_found: 'That entity no longer exists.',
not_owner: 'You don\'t own this entity, so you can\'t change it.',
};
function humanizeError(detail) {
if (!detail) return 'Something went wrong. Please try again.';
// Structured detail (FastAPI wraps the dict at .detail). Two shapes:
// {code: "submission_blocked", submission_id, checks: {...}} — guardrails
// {code: "quota_exceeded", limit, blocked_in_last_24h, hint} — quota
// Drop the bare ``Upload failed: [object Object]`` fallback.
if (typeof detail === 'object') {
const code = detail.code || '';
if (code === 'submission_blocked') {
const lines = ['Upload blocked by automated checks.'];
const checks = detail.checks || {};
const inline = (checks.static_security && checks.static_security.findings) || [];
for (const f of inline.slice(0, 5)) {
const where = (f.file || '?') + ':' + (f.line || '?');
lines.push('• ' + where + ' — ' + (f.reason || f.category || 'security finding'));
}
if (inline.length > 5) lines.push(' …and ' + (inline.length - 5) + ' more.');
const manifestIssues = (checks.manifest && checks.manifest.issues) || [];
for (const m of manifestIssues.slice(0, 3)) {
lines.push('• manifest: ' + m);
}
lines.push('');
lines.push('Fix the issues above and re-upload, or open the admin queue if the rule was over-strict.');
return lines.join('\n');
}
if (code === 'quota_exceeded') {
return 'Upload blocked: too many rejected uploads in the last 24 hours '
+ '(' + (detail.blocked_in_last_24h || '?') + '/' + (detail.limit || '?') + '). '
+ (detail.hint || 'Wait for the window to reset.');
}
if (code) return 'Upload failed: ' + code;
// Unknown object shape — last resort, JSON-stringify so the user
// sees the raw payload instead of [object Object].
try { return 'Upload failed: ' + JSON.stringify(detail); }
catch (_) { return 'Upload failed.'; }
}
const s = String(detail);
if (ERROR_MESSAGES[s]) return ERROR_MESSAGES[s];
// Codes with a parameter (e.g. "unknown_type:foo")
const head = s.split(':', 1)[0];
if (ERROR_MESSAGES[head]) return ERROR_MESSAGES[head];
// Last resort — show a friendly wrapper plus the raw token.
return 'Upload failed: ' + s;
}
function showError(msg) {
banner.className = 'banner error';
banner.firstElementChild.textContent = '!';
bannerText.textContent = msg;
banner.hidden = false;
}
function clearBanner() { banner.hidden = true; }
function showStep(n) {
document.getElementById('step-1').classList.toggle('is-active', n === 1);
document.getElementById('step-2').classList.toggle('is-active', n === 2);
document.getElementById('seg-2').classList.toggle('is-done', n >= 2);
clearBanner();
window.scrollTo({top: 0, behavior: 'smooth'});
}
// Type tiles
document.querySelectorAll('#type-tiles label').forEach(lbl => {
lbl.addEventListener('click', () => {
document.querySelectorAll('#type-tiles label').forEach(l => l.classList.remove('is-active'));
lbl.classList.add('is-active');
});
});
// Generic drop-zone wiring — used for both the ZIP archive and the cover
// photo. The two differ only in their accepted MIME/extension and the
// onAccept callback that updates page state.
function wireDropZone(dropEl, fileInput, pickBtn, validate, onAccept) {
// Click anywhere on the zone (except the explicit button — which already
// fires its own click) opens the file picker.
pickBtn.addEventListener('click', (e) => { e.stopPropagation(); fileInput.click(); });
dropEl.addEventListener('click', (e) => {
if (e.target.tagName !== 'BUTTON') fileInput.click();
});
fileInput.addEventListener('change', () => {
if (fileInput.files[0]) handle(fileInput.files[0]);
});
function handle(f) {
const err = validate(f);
if (err) { showError(err); return; }
// Sync the <input> so the form-submit fallback works too.
try {
const dt = new DataTransfer();
dt.items.add(f);
fileInput.files = dt.files;
} catch (_) {}
dropEl.classList.remove('is-dragover');
onAccept(f);
clearBanner();
}
['dragenter', 'dragover'].forEach(evt => {
dropEl.addEventListener(evt, (e) => {
e.preventDefault();
e.stopPropagation();
if (e.dataTransfer) e.dataTransfer.dropEffect = 'copy';
dropEl.classList.add('is-dragover');
});
});
['dragleave', 'dragend'].forEach(evt => {
dropEl.addEventListener(evt, (e) => {
// dragleave fires when entering a child; only un-highlight when the
// pointer actually leaves the drop zone.
if (evt === 'dragleave' && dropEl.contains(e.relatedTarget)) return;
dropEl.classList.remove('is-dragover');
});
});
dropEl.addEventListener('drop', (e) => {
e.preventDefault();
e.stopPropagation();
dropEl.classList.remove('is-dragover');
const file = e.dataTransfer && e.dataTransfer.files && e.dataTransfer.files[0];
if (file) handle(file);
});
}
// Page-level guard: dropping anywhere outside the zones shouldn't navigate
// away from the page (browser default behavior for files).
['dragover', 'drop'].forEach(evt => {
window.addEventListener(evt, (e) => { e.preventDefault(); });
});
// ZIP zone
wireDropZone(
zipDrop, zipInput, zipPick,
(f) => /\.zip$/i.test(f.name) ? null : 'Only .zip files are accepted.',
(f) => {
zipFile = f;
zipDrop.classList.add('has-file');
zipLabel.innerHTML = 'Selected:<span class="filename">' + escapeHtml(f.name) + '</span>';
zipMeta.textContent = formatSize(f.size);
},
);
// Photo zone — accepts JPG / PNG / WebP up to 5 MB.
const PHOTO_MAX = 5 * 1024 * 1024;
wireDropZone(
photoDrop, photoInput, photoPick,
(f) => {
if (!/^image\/(jpeg|png|webp)$/i.test(f.type) &&
!/\.(jpe?g|png|webp)$/i.test(f.name)) {
return 'Photo must be a JPG, PNG, or WebP image.';
}
if (f.size > PHOTO_MAX) {
return 'Photo too large — max 5 MB.';
}
return null;
},
(f) => {
photoDrop.classList.add('has-file');
photoLabel.innerHTML = 'Selected:<span class="filename">' + escapeHtml(f.name) + '</span>';
},
);
// Docs
// v32: client-side allowlist sanity check. The same allowlist is enforced
// server-side (returns HTTP 415), so this is purely UX — give the user a
// useful inline message before they submit instead of a generic error after
// the round-trip.
const ALLOWED_DOC_EXT = new Set(['.pdf', '.md', '.markdown', '.txt']);
addDocBtn.addEventListener('click', () => docInput.click());
docInput.addEventListener('change', () => {
for (const f of docInput.files) {
const lower = (f.name || '').toLowerCase();
const dot = lower.lastIndexOf('.');
const ext = dot >= 0 ? lower.slice(dot) : '';
if (!ALLOWED_DOC_EXT.has(ext)) {
alert(
'Unsupported document format: "' + f.name + '". '
+ 'Only PDF (.pdf), Markdown (.md, .markdown), and plain text (.txt) are accepted.'
);
continue;
}
docs.push(f);
}
docInput.value = '';
renderDocs();
});
function renderDocs() {
docList.innerHTML = '';
docs.forEach((f, i) => {
const item = document.createElement('div');
item.className = 'doc-item';
item.innerHTML = `
<span class="name">${escapeHtml(f.name)}</span>
<span class="size">${formatSize(f.size)}</span>
<button type="button" data-i="${i}" title="Remove">✕</button>`;
docList.appendChild(item);
});
docList.querySelectorAll('button[data-i]').forEach(btn => {
btn.addEventListener('click', () => {
docs.splice(parseInt(btn.dataset.i, 10), 1);
renderDocs();
});
});
}
// Step 1 → preview
nextBtn.addEventListener('click', async () => {
clearBanner();
if (!zipFile) { showError('Please choose a ZIP file.'); return; }
const type = document.querySelector('input[name=type]:checked').value;
nextBtn.disabled = true;
nextBtn.textContent = 'Validating…';
try {
const fd = new FormData();
fd.append('file', zipFile);
fd.append('type', type);
const res = await fetch('/api/store/entities/preview', {method: 'POST', body: fd});
if (!res.ok) {
let msg = 'Validation failed.';
try {
const j = await res.json();
if (j.detail) msg = humanizeError(j.detail);
} catch(_) {}
showError(msg);
return;
}
const preview = await res.json();
document.getElementById('name').value = preview.name || '';
document.getElementById('description').value = preview.description || '';
showStep(2);
} catch (err) {
showError(String(err));
} finally {
nextBtn.disabled = false;
nextBtn.textContent = 'Next →';
}
});
backBtn.addEventListener('click', () => showStep(1));
// Step 2 → final create
finishBtn.addEventListener('click', async () => {
clearBanner();
const name = document.getElementById('name').value.trim();
if (!name) { showError('Name is required.'); return; }
const type = document.querySelector('input[name=type]:checked').value;
finishBtn.disabled = true;
finishBtn.textContent = 'Uploading…';
try {
const fd = new FormData();
fd.append('file', zipFile);
fd.append('type', type);
fd.append('name', name);
fd.append('description', document.getElementById('description').value);
const cat = document.getElementById('category').value;
if (cat) fd.append('category', cat);
const vurl = document.getElementById('video_url').value.trim();
if (vurl) fd.append('video_url', vurl);
const photo = document.getElementById('photo').files[0];
if (photo) fd.append('photo', photo);
for (const d of docs) fd.append('docs', d);
const res = await fetch('/api/store/entities', {method: 'POST', body: fd});
if (res.ok) {
const entity = await res.json();
window.location = `/marketplace/flea/${encodeURIComponent(entity.id)}`;
return;
}
// Inline-blocked uploads (422 with submission_blocked) now persist
// the entity at visibility=hidden so admins can rescan/override and
// submitters can see the full quarantine banner. Redirect to the
// detail page when the response carries an entity_id — same UX as
// a successful upload, just landing on a banner instead of an
// approved card.
let msg = 'Upload failed.';
try {
const j = await res.json();
const eid = j && j.detail && j.detail.entity_id;
if (eid && j.detail.code === 'submission_blocked') {
window.location = `/marketplace/flea/${encodeURIComponent(eid)}`;
return;
}
if (j.detail) msg = humanizeError(j.detail);
} catch(_) {}
showError(msg);
} catch (err) {
showError(String(err));
} finally {
finishBtn.disabled = false;
finishBtn.textContent = 'Finish';
}
});
function escapeHtml(s) {
return String(s).replace(/[&<>"']/g, ch => ({'&':'&amp;','<':'&lt;','>':'&gt;','"':'&quot;',"'":'&#39;'}[ch]));
}
function formatSize(n) {
if (n < 1024) return n + ' B';
if (n < 1024 * 1024) return (n / 1024).toFixed(1) + ' KB';
return (n / 1024 / 1024).toFixed(1) + ' MB';
}
</script>
{% endblock %}
Reference in a new issue View git blame Copy permalink