> ## Documentation Index
> Fetch the complete documentation index at: https://docs.theaitracker.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Visibility pipeline

> From an active prompt to a visibility score.

## The path a prompt takes

<Steps>
  <Step title="A batch is dispatched">
    `weekly:prompt-batch` collects every `active` prompt across active campaigns and
    dispatches `WeeklyPromptBatchJob`. Despite the name, the schedule runs **daily** at
    03:00 UTC — the command name was kept for compatibility when the cadence changed.
  </Step>

  <Step title="Each provider is fetched">
    One `Fetch*` job per provider issues the prompt: OpenAI, Gemini, Claude, Perplexity and
    Grok through their APIs, and Google AI Overview / AI Mode through SerpApi
    (`ProcessSerpApiOverviewJob`, `ProcessSerpApiAiModeJob`). Every response is written as a
    `PromptRun` tagged with a shared `batch_id`.
  </Step>

  <Step title="Answers are parsed">
    The matching `Process*AnswersJob` runs `VisibilityDetector` over the answer text to
    decide whether the brand appeared and in what form, setting the run's visibility fields.
    Brand variants generated during onboarding are what make this robust to the brand being
    referred to by a shortened or alternative name.
  </Step>

  <Step title="Competitors are extracted">
    Domains mentioned alongside the brand become `DiscoveredCompetitor` and
    `CompetitorAppearance` rows. `ClassifyBatchCompetitorsJob` (or `competitors:classify`)
    then asks an LLM which of them are genuine direct competitors.
  </Step>

  <Step title="Scores are calculated">
    `competitors:score` runs `IndustryVisibilityScorer`, applying the weighted formula in
    `config/scoring.php` to produce the industry visibility score shown on `/visibility`.
  </Step>

  <Step title="Screenshots are captured">
    `TakeSearchScreenshotJob` stores a SERP screenshot per run through ScreenshotOne, exposed
    on the model as `screenshot_url`.
  </Step>
</Steps>

## Running it by hand

Waiting for 03:00 UTC is rarely what you want while developing. Each stage has a manual
entry point:

```bash theme={null}
# Full batch across all active prompts, ignoring the recency guard
php artisan weekly:prompt-batch --force

# SerpApi only, with debug logging — scoped to one prompt or campaign
php artisan dispatch:serpapi --prompt=123
php artisan dispatch:serpapi --campaign=45 --force

# Classification and scoring
php artisan competitors:classify
php artisan competitors:score

# Sentiment for a single campaign (takes the campaign UUID)
php artisan sentiment:fetch-daily --campaign=<uuid>
```

<Warning>
  These commands spend real money on provider APIs. `--force` bypasses the check that
  prevents re-running prompts that were processed recently, so it will re-bill every prompt
  in scope. Scope to a single campaign or prompt while developing.
</Warning>

A batch can also be triggered from the UI per campaign via
`POST /campaigns/{campaign}/generate-prompts`, and
`GET /campaigns/{campaign}/generation-prompt-preview` shows the exact generation prompt that
would be sent — useful when tuning prompt generation without paying for a run.

## Sentiment pipeline

Sentiment is a separate daily flow (`sentiment:fetch-daily`, 01:00 UTC) that dispatches
`FetchCampaignSentiment` and per-provider jobs (`FetchOpenAiSentimentJob`,
`FetchGeminiSentimentJob`, `FetchGrokSentimentJob`, `FetchPerplexitySentimentJob`). It runs
through `Services/BrandSentiment`:

1. `TopicSelector` picks which topics to evaluate this cycle
2. `TopicDataRetriever` gathers answers using `RetrievalPromptWriter`
3. `TopicSentimentScorer` scores mentions using `EvaluationPromptWriter`
4. `BrandSentimentFinalizer` writes `SentimentRecord` and `SentimentMention` rows

The UI polls `GET /api/sentiment/status` while this is in flight.

## Site intelligence

Independent of prompts, several jobs profile the tracked domain itself:

| Job / command                | What it collects                                  |
| ---------------------------- | ------------------------------------------------- |
| `FetchSiteInfoJob`           | Site metadata via `SiteInfoService`               |
| `FetchPageSpeedInsightsJob`  | Core Web Vitals via PageSpeed Insights            |
| `CheckAiTrainingPresenceJob` | Whether the domain appears in AI training corpora |
| `bot-access:refresh`         | `robots.txt` rules for AI crawlers (monthly)      |

## Cost tracking

Every provider call goes through `TokenUsageTracker`, which writes an `LlmUsage` row.
`config/ai.php` carries `cost_per_1k_tokens` per model, so spend is derived from recorded
token counts rather than fetched from the providers.