# Sources Maintenance Rules

This directory is the evidence layer for the onboarding kit. Modules turn evidence into guided steps; sources preserve the original context that RAG-style answers can cite.

## Directory Contract

- `source-manifest.yaml`: required inventory of every source, asset, extracted text file, status, sensitivity, and retrieval time.
- `raw/`: full or curated text from primary sources and sanitized manual notes.
- `linked-docs/`: fetched text from nested Lark docs, wiki pages, or docx pages.
- `pdf-text/`: text extracted from downloaded PDF files.
- `image-notes/`: searchable image alt text, OCR notes, or manual descriptions. Do not assume image binaries are searchable.
- `tables/`: exported CSV and markdown versions of embedded sheets or tables.
- `external-links/`: fetched or metadata-only notes for public external links.
- `../assets/pdf/`, `../assets/images/`, `../assets/video/`: downloaded binary originals. These are not searched directly.

## Required Manifest Fields

Each source item in `source-manifest.yaml` must include:

- `id`: stable lowercase identifier.
- `type`: one of `lark_doc`, `linked_lark_doc`, `embedded_sheet`, `embedded_media_pdf`, `embedded_media_video`, `embedded_image`, `external_link`, `manual_note`, or `self_reference`.
- `title`: human-readable source title.
- `status`: `ingested`, `downloaded_with_alt_text`, `downloaded_metadata_only`, `downloaded_no_text`, `metadata_only`, `failed`, or `skipped_self_reference`.
- `sensitivity`: `public`, `internal`, or `confidential`.
- `retrieved_at`: ISO timestamp.
- `text_path` or `csv_path`: required when the source should be searchable.
- `asset_path`: required when a binary file was downloaded.
- `notes`: required when the status is not fully `ingested`.

## Adding Sources

Preferred path:

1. Add the source to the primary Feishu document or place the original file in the matching `assets/` folder.
2. Run `python3 scripts/ingest_lark_sources.py` from the kit root.
3. Run `python3 scripts/validate_kit.py`.
4. Run `python3 scripts/index_knowledge.py`.
5. Verify with `python3 scripts/search_knowledge.py "<query>"`.

Manual path for non-Lark material:

1. Put binary originals under `assets/pdf/`, `assets/images/`, or `assets/video/`.
2. Put searchable extracted text under `sources/pdf-text/`, `sources/image-notes/`, `sources/tables/`, or `sources/external-links/`.
3. Add or update the matching item in `source-manifest.yaml`.
4. Re-run validation and indexing.

## Non-Negotiable Rules

- Do not use unmanifested materials as answer evidence.
- Do not cite a binary asset unless a corresponding searchable text file exists.
- Do not store credentials, verification codes, private keys, identity numbers, bank card numbers, or raw signed Feishu asset URLs in searchable text.
- If a source cannot be downloaded or parsed, keep a `metadata_only` or `failed` manifest item with the original title, context, and next action.
- If a source changes substantially, re-run ingestion and indexing before answering from it.
