# Scout Chrome Crawler

Private Playwright service for Scout `dev_mode=true`. It is not a public API.
It binds to `127.0.0.1` only, requires a bearer token, and receives decrypted
Chrome storage state only from the backend worker/gateway process.

## Install

```bash
cd apps/backend/crawler
npm install
npx playwright install --with-deps chromium
```

## Configure

Set the same private token in the local-only `apps/backend/etc/gateway.yaml`:

```yaml
Scout:
  SessionSecret: "a dedicated long random secret"
  CrawlerEndpoint: "http://127.0.0.1:8891"
  CrawlerToken: "a different long random token"
```

Start the crawler with the matching token:

```bash
cd apps/backend/crawler
SCOUT_CRAWLER_TOKEN='the CrawlerToken value' npm run start
```

Optional: `SCOUT_CRAWLER_PORT=8891` changes the local port.

## Runtime Rules

- Never expose this port through nginx, Docker host publishing, or the public internet.
- Never place `CrawlerToken`, `SessionSecret`, or storage state in a tracked file or log.
- `dev_mode=false` does not use this service; it uses the configured API provider.
- `dev_mode=true` requires a freshly synchronized Chrome session. Missing or expired sessions fail with a user-actionable error.
- The service supports `/v1/threads/search` and `/v1/threads/resolve`; both require `Authorization: Bearer <CrawlerToken>`.

## Operations

1. Start Mongo, Redis, gateway, worker, and this crawler service.
2. In Harbor Desk settings, enable dev mode and synchronize Chrome Threads session.
3. Start a Scout run. The worker chooses crawler mode and contacts only `CrawlerEndpoint`.
4. If Threads redirects to login, sync the session again. No fallback to API occurs in dev mode.