# 3D Asset Pipeline

Al-Amr environments are predominantly procedural and hold to strict performance budgets. When a world or a Plugin does need real 3D models — foliage, curated props, avatars — they pass through this pipeline.

This document covers the journey of a 3D asset from a downloaded file to GPU memory.

## 1. Packing

There is one packer, `packages/cli/assets/pack-assets.mjs`, and every Environment runs it the same way — the first-party worlds in this repository and an independent project scaffolded by `npm create al-amr@latest` alike. `alamr create environment` copies it into a generated project as `scripts/pack-assets.mjs`.

The command takes no required arguments, because the directory layout is the contract:

```bash
npm run assets:pack
```

Every `.glb` or `.gltf` inside `assets/models/<pack>/` becomes one named variant of `public/models/PACK-<pack>.glb`, where the variant name is the source file name without its extension. Source models are only ever read: nothing under `assets/models/` is created, modified, or deleted, because a downloaded model has no undo. A pack whose sources have not changed since its output was written is skipped; `--force` rebuilds it anyway.

Each pack prints a table of its variants with primitive counts, triangle counts, and a bounding box in metres, and `--json` prints the same measurements as structured data for an agent. The bounding box is the load-bearing column: a model exported in centimetres arrives a hundred times too small and renders perfectly, invisibly.

### What the packer does

1. **Merging.** Every model in the folder is merged into one document, so N HTTP requests become one.
2. **Deduplication (`dedup()`).** A nature set's ten trees that all reference `Bark_Normal.png` collapse to a single texture uploaded once. This is the reason packs exist at all: ten self-contained `.glb` files would mean ten downloads and — the expensive half — ten distinct GPU textures, because nothing tells the renderer they are the same image.
3. **Joining (`join()`).** Same-material primitives within a variant are merged, so an instanced variant costs one draw call per material rather than one per authored mesh. It runs after `dedup`/`prune` and before `weld`/`meshopt`, so it never operates on an already-compressed buffer.
4. **Texture compression (KTX2 / Basis Universal).** Base colour and emissive maps use **ETC1S** (perceptual, highly compressed); normal and ORM maps use **UASTC** (linear, preserves vector data). Encoding a normal map as ETC1S quantises it in a colour-perceptual space, and a normal map is not a colour but a vector field — lighting goes blotchy. Textures are resized against their own dimensions, not against a magic constant.
5. **Material optimization.** `--alpha-mask` converts expensive `BLEND` materials to `MASK`, eliminating back-to-front GPU sorting — which matters most for instanced foliage, where per-instance sorting is exactly what makes transparent geometry a poor instancing candidate.
6. **Geometry compression (Meshopt).** Vertex and index buffers are compressed with Meshoptimizer.

### Why first-party Environments invoke it by path

Each Environment declares `"assets:pack": "node ../../packages/cli/assets/pack-assets.mjs"`. The packer's seven dependencies are root `devDependencies` and are deliberately not redeclared per Environment, and no wrapper script at the repository root is needed either.

That works because Node resolves an ES module's imports from **the script's own location**, never from `process.cwd()`. The lookup walks `packages/cli/assets/node_modules` → `packages/cli/node_modules` → `packages/node_modules` → the repository root, where all seven live. `environments/<env>/node_modules` is never consulted, so pnpm's strict per-package linking — which does not expose root `devDependencies` to a workspace package — never comes into play. Verified by running the packer with the working directory set to `environments/park`: all seven resolved, and none of them exist in that Environment's own `node_modules`.

The generated-project copy needs no such reasoning: it sits at `scripts/pack-assets.mjs` inside the project that declares the seven dependencies itself.

Packing is never wired into `predev` or `prebuild`. A build must not go looking for `assets/models/` or start a texture encoder; the command is always explicit.

> The six packs already in `environments/park/public/models/` were produced by an earlier repository-only packer that this pipeline replaced. Their raw sources are not in the repository, so they have not been rebuilt — rebuilding them would churn bytes without changing what ships.

## 2. Sourcing and preparing

All 3D models used in the core repositories must be **CC0** (public domain) or compatibly licensed. Providers like Quaternius are a frequent source.

- Source models should be `.glb` or `.gltf`.
- Avoid pre-optimized or obfuscated formats during authoring.
- Pivot points (origin) belong at the logical base of the object, especially for anything that will be instanced or driven by physics or wind shaders.
- For modular assets — a tree with separate trunk and canopy meshes — keep the local transforms correct relative to the asset root.

## 3. Engine integration: `useGltfPack`

As set out in [ADR-0079](../decisions/ADR-0079-shared-self-hosted-gltf-asset-loading.md), `useGLTF` from `drei` is not used: it fetches decoders from external CDNs (gstatic, jsDelivr), which breaks both self-hosting and the performance budgets.

Instead, the `useGltfPack` hook from `@al-amr/r3f`:

```typescript
import { useGltfPack } from "@al-amr/r3f";

const pack = useGltfPack("/models/PACK-BirchTree.glb");
const variant = pack.getVariant("BirchTree_1");
```

### Instancing

To keep draw calls to a minimum, extract the geometry and materials from the loaded pack and instance them with `InstancedMesh`.

```typescript
// Example: Creating an InstancedMesh from a variant
const instanced = new InstancedMesh(mesh.geometry, mesh.material, placementCount);
```

### Ownership and cloning

Whatever `getVariant(name)` returns belongs to the pack cache, which owns its geometries and textures and frees them when the last consumer unmounts. Clone it with `.clone(true)` before it enters a scene: an `Object3D` has exactly one parent, so mounting the same variant object in two places silently removes it from the first, with no error and no warning. Past roughly twenty copies of one variant, stop cloning and instance instead.

The full set of rules generated scene code must follow is the [agent recipe](/docs/recipes/environment-3d-models); the human path from downloading a model to seeing it in a world is the [3D models guide](/docs/environment/3d-models).

## 4. TSL material manipulation and wind shaders

When rendering vegetation — trees, grass, bushes — GPU wind displacement is applied with Three.js Shading Language (TSL).

### The local matrix problem

A common failure when applying vertex displacement to glTF assets is stretching, or no movement at all. It happens because `positionGeometry.y` measures a vertex's height relative to the origin of _that specific mesh_.

If an artist authored a tree's leaves as a separate mesh at `y = 3.0` metres, the `positionGeometry.y` of the leaves themselves still starts at `0.0`. A wind shader that assumes `0.0` is the trunk base applies no wind to the leaves.

### The solution: `rootY`

Calculate the vertex's absolute height relative to the root of the whole asset. Extract the sub-mesh's local transform and pass it to the TSL material generator:

```typescript
// 1. Extract the local offset and scale of the mesh relative to the variant root
const localPosition = new Vector3();
const localQuaternion = new Quaternion();
const localScale = new Vector3();
localMatrix.decompose(localPosition, localQuaternion, localScale);

// 2. Calculate the true height from the root
const rootY = positionGeometry.y.mul(float(localScale.y)).add(float(localPosition.y));

// 3. Calculate sway weight based on the true height
const swayWeight = pow(
  clamp(rootY.sub(float(swayBaseY)).div(float(swayTopY - swayBaseY)), 0, 1),
  float(swayPower),
);
```

### Arc-preserving displacement

Rather than moving vertices linearly in the XZ plane, which stretches the geometry, apply a quadratic Y-drop to preserve the arc length of the bending plant:

```typescript
const displaced = positionLocal.add(
  vec3(
    float(WIND_DIRECTION_XZ[0]).mul(amplitude),
    amplitude.mul(amplitude).mul(-0.5), // Quadratic drop preserves arc length
    float(WIND_DIRECTION_XZ[1]).mul(amplitude),
  ),
);
mat.positionNode = displaced;
```

Foliage then sways without distorting the mesh proportions.