# Syncing data

## How syncing works

Each submodule fetches data from the Blizzard API and stores it locally in Drupal. There are two sync patterns:

### Catalog sync (reference data)

Realms, classes, races, specializations, power types, and reputation factions are synced by fetching an index endpoint from Blizzard, then creating or updating a local taxonomy term for each item.

These syncs use **conditional requests** — they send `If-Modified-Since` from the previous sync. If Blizzard responds with "304 Not Modified", no data is written and the operation completes instantly. Use `--force` to bypass this.

### Queue-based sync

Large catalogs — achievements, mounts, pets, titles, toys — are synced one entity per Advanced Queue job rather than in a single request. The module enqueues the jobs; the queue processor (cron or a dedicated worker) drains them. On the dashboard, **Sync now** drains the queue inline as a Drupal batch with a progress bar.

## Sync vs refresh

- **Sync** = update a catalog or reference-data vocabulary from its Blizzard index endpoint. Affects the "what exists" dictionary; never touches per-character data directly.
- **Refresh** (characters and guilds) = re-fetch a single profile from `/profile/wow/character/{realm}/{name}` or the guild equivalent, update the owning entity, and fire registered `CharacterDataProvider` / `GuildDataProvider` plugins so their per-owner rows land too.

Both flow through the same `BattleNetClient`, so both count against rate limits.

## Recommended sync order

For a fresh installation, sync in this order:

1. **Reference data** — realms, classes, races, specializations, power types, reputation factions
2. **Achievement catalog** — categories, then achievements
3. **Collection catalogs** — mounts, pets, titles, toys
4. **Characters and guilds** — imported individually with `wow:character-lookup` / `wow:guild-lookup`; their collection and reputation data is pulled automatically by registered providers.

Characters reference realms, classes, and races; collection tracking references the catalog entities. Syncing in this order avoids temporary gaps.

## Automating with cron

For ongoing maintenance:

- **Cron** processes the Advanced Queue automatically and runs the [TTL sweep](#data-freshness-and-the-30-day-rule) at most once every 23 hours.
- **Scheduled drush commands** re-sync reference data periodically (weekly or monthly is sufficient — this data rarely changes).
- **Character refreshes** happen on demand (e.g., when a player visits their profile) or via a scheduled task you wrap around `CharacterSync::refresh()`. There is no built-in bulk-refresh command.

A `CharacterSync::refresh()` that hits a 403 / 404 from Blizzard returns `NULL` rather than deleting the entity; stale characters are cleaned up by the TTL sweeper, not by the refresh path.

## Rate limiting

The module enforces both Blizzard rate-limit ceilings (36,000 requests per hour and 100 per second) across every PHP process (web, drush, queue workers). Counters are stored in Drupal State and check+increment is serialized under a lock. If a request would exceed either ceiling, `BattleNetClient::get()` throws a `RuntimeException` — queue workers naturally pace themselves through the Advanced Queue processor, and ad-hoc drush calls surface the exception to the operator. See [API compliance](../development/api-compliance.md#rate-limiting).

## Data freshness and the 30-day rule

Blizzard's API Terms of Use require all cached data to be refreshed or deleted within 30 days. Two freshness fields exist for technical reasons:

- **Content entities** (characters, guilds, achievements, mounts, pets, titles, toys, items, and their per-owner junction rows) use a `last_fetched` base field.
- **Taxonomy reference data** (realms, classes, races, specializations, power types, reputation factions) uses a `wow_last_fetched` configurable field — Drupal's `taxonomy_term` entity can't host arbitrary base fields, so submodules ship it as config.

The `wow.ttl_sweeper` service runs daily under cron (`hook_cron` throttled to once per 23 hours) and deletes content entities whose `last_fetched` exceeds 30 days, in chunks of 50 to keep memory bounded. Both catalog entities (mounts, pets, titles, etc.) and per-owner junction rows (collected mounts, achievement progress, etc.) are swept independently — junctions need their own TTL because Blizzard's API may return a partial collection on a character refresh and because un-collected items disappear from the API but linger locally until swept.

Reference data is **not** swept — operators keep it fresh with the scheduled sync commands (`drush wow:sync-realms`, etc.). For a one-shot sweep:

```
drush wow:sweep-ttl
```

See [API compliance](../development/api-compliance.md#30-day-data-ttl) for the full contract.
