agnes-the-ai-analyst/connectors
ZdenekSrotyr 10d7bd62f8 fix(bq): #160 wrap views via bigquery_query() for VIEW/MATERIALIZED_VIEW
Issue #160: da query --remote against query_mode='remote' BQ rows whose
underlying entity is a VIEW or MATERIALIZED_VIEW returned a DuckDB catalog
error because the extractor (with legacy_wrap_views=False default since
the v2 fetch primitives release) skipped master-view creation for those
entity types — but kept inserting the _meta row, leaving operators with a
registered name that resolves to nothing.

Always create a master view for entity types we have proven runtime support
for in this codebase:

  BASE TABLE          → bq."<dataset>"."<source_table>"
                        (Storage Read API path; predicate pushdown)
  VIEW / MAT_VIEW     → bigquery_query('<project>', 'SELECT * FROM `proj.ds.tbl`')
                        (jobs API path; no pushdown — the upcoming /api/query
                        cost guardrail bounds the scan; was the legacy
                        legacy_wrap_views=True branch SQL form, just always-on)

For other entity types (EXTERNAL, SNAPSHOT, CLONE, future), log a warning
and SKIP both the master view AND the _meta row. The registry row remains
intact so /api/v2/scan still works for `da fetch`; we just don't expose a
stale _meta entry that the orchestrator would later strand.

The legacy_wrap_views config knob is still readable in this commit (read
returns the value, which is then ignored). Removal across the rest of
the codebase happens in the follow-up REFACTOR commit.

tests/test_bigquery_extractor.py:
- Add 3 RED tests covering the new always-wrap behavior:
  test_view_creates_wrap_view_with_default_config,
  test_materialized_view_creates_wrap_view_with_default_config,
  test_unsupported_entity_type_skips_meta_and_view.
- Fix pre-existing flakiness in test_main_exits_when_project_missing
  by resetting app.instance_config cache before the no-project mock —
  the prior test populates the cache with a project, and removing the
  legacy_wrap_views get_value() call surfaced this latent ordering bug.
2026-05-04 10:31:35 +02:00
..
bigquery fix(bq): #160 wrap views via bigquery_query() for VIEW/MATERIALIZED_VIEW 2026-05-04 10:31:35 +02:00
jira feat(observability): request_id end-to-end + dev debug toolbar + centralized logging (#136) 2026-04-29 22:54:21 +02:00
keboola fix(materialized): address 4 Devin Review findings on PR #152 2026-05-01 20:58:17 +02:00
llm Suppress httpcore debug logging in LLM connector 2026-03-23 12:57:35 +01:00
openmetadata feat(scheduler): re-wire sync_schedule + script.schedule; tune via env; OpenMetadata TLS (#135) 2026-04-29 22:06:30 +02:00
__init__.py Extract Jira into connectors/jira module 2026-03-09 11:17:50 +01:00