# BTV Innsbruck - Crawler-to-Index Report

## Purpose

Reference report showing the full data pipeline for a single project page
(BTV Innsbruck - Stadtforum). Use this to understand what the crawler produces,
how the pipeline processes it, what gets stored in SQLite, and what ends up
in the FAISS vector index.

## Data Pipeline

```
crawler (external)
  -> .html, .md, .txt, .crawl.json, .images.json, _removed_parts.md
pipeline_v2.py (DeepSeek text analysis)
  -> pages, page_relations, page_entities, page_questions, page_features
  -> vector_embeddings (bge-m3 1024-dim)
image_grid_threaded.py (Gemini image classification)
  -> images.analysis_status, images.image_category, images.ai_description
  -> image_summaries.summary
build_faiss.py
  -> online_knowledge.faiss + online_knowledge_meta.json
mcp_backend_knowledge.py (search at runtime)
  -> FAISS vector search + SQLite enrichment
```

## Page: BTV Innsbruck - Stadtforum

- URL: https://www.prolicht.at/de/inspiration/projekte/btv-innsbruck/
- Stem: `prolicht__at__de__inspiration__projekte__btv-innsbruck`
- page_id: 13
- page_category: `project_reference`
- segment: `office`
- content_quality_score: 9

## Source Files (in zip)

| File | Description |
|------|-------------|
| `crawled/btv-innsbruck.md` | Crawler markdown output (main content) |
| `crawled/btv-innsbruck.txt` | Crawler plain text output (includes nav/footer) |
| `crawled/btv-innsbruck.crawl.json` | Crawler metadata (URLs, screenshot info) |
| `crawled/btv-innsbruck.images.json` | Image manifest (11 images with URLs, local paths, sizes) |
| `crawled/btv-innsbruck_removed_parts.md` | Boilerplate stripped by crawler (nav, footer, color world sidebar) |
| `crawled/projects-listing-page.md` | The project INDEX page for comparison (contains filter sidebar with ALL product families) |
| `code/pipeline_v2.py` | Text analysis pipeline (DeepSeek + bge-m3 embedding) |
| `code/mcp_backend_knowledge.py` | Runtime FAISS search backend |
| `code/models.py` | Pydantic schemas for PageAnalysis |
| `code/build_faiss.py` | FAISS index builder |
| `db-report/BTV-INNSBRUCK-DB-REPORT.txt` | Full SQLite dump for this page |

## Key Indexed Data

### Product Families (from relations)
GLORIOUS, HADI, HYPRO, OIKO, SLIDER, SUPER-G

### Architects
Heinz Tesar (design), Obermoser arch-omo (realisation)

### Images (10 classified, 1 logo skipped)
- 4x product_image (isolated product shots: glorious, hadi, superg, oiko)
- 2x reference_project_image (architectural photos of the building)
- 2x product_detail_image (hypro, slider close-ups)
- 2x product_application_image (office/meeting room installed lighting)
- 1x logo (skipped)

### Vector Embeddings (5 documents)
- page_summary, page_questions, page_entities, page_relations, project_profile

## Known Issues

1. **HADI vs HADI-M**: The product family section says "HADI" but the image
   alt text on line 114 says "HADI-M LED Pendelleuchte...". The LLM extracted
   "HADI" (correct per the family link), so `target_normalized = "hadi"`.
   A user searching for "HADI-M" (normalized to "hadi_m") won't match.

2. **Linked pages are footer noise**: page_links contains only footer/nav
   links (Alle Produkte, Produktfamilien, Downloads). The product family
   card links (SUPER-G -> produktfamilien/super-g) are NOT captured.

3. **Duplicate image filenames**: `office-buero-wand-beleuchtung-leuchte-slider-hypro.jpg`
   appears twice (hero webp + gallery thumb jpg) with different sizes.

4. **No synonyms stored**: The page has no synonym entries despite obvious
   candidates (BTV = Bank für Tirol und Vorarlberg, Stadtforum = City Forum).

5. **read_page_markdown()** strips frontmatter, logo, and social links but
   the _removed_parts.md content (nav, footer, color world) is already
   separated by the external crawler. The ## Inhalt / ## Seitentools markers
   only exist in _removed_parts.md, never in the main .md.
