# AnySurface Architecture

## Overview

AnySurface enables interactive projected surfaces by combining camera input, projector output, and laser pointer detection. The system maps the relationship between camera pixels and projector pixels, then uses that mapping to convert laser pointer positions into DOM mouse events.

## System Diagram

```
                    ┌─────────────────────┐
                    │      Surface        │  (Main Orchestrator)
                    └─────────┬───────────┘
          ┌───────────┬───────┼───────┬──────────┐
          ▼           ▼       ▼       ▼          ▼
     ┌─────────┐ ┌───────┐ ┌─────┐ ┌────────┐ ┌──────────────┐
     │GrayScan │ │ Align │ │Shut-│ │Camera- │ │Input-        │
     │         │ │       │ │ter- │ │Inter-  │ │Simulator     │
     │         │ │       │ │Est. │ │face    │ │              │
     └────┬────┘ └───────┘ └─────┘ └───┬────┘ └──────┬───────┘
          │                             │              │
          ▼                             │              ▼
   ┌──────────────┐                     │     ┌──────────────────┐
   │Correspondence│                     │     │PointerDetector-  │
   │              │                     │     │Client             │
   └──────┬───────┘                     │     └────────┬─────────┘
          │                             │              │
          ▼                             │              ▼
   ┌─────────────┐              ┌───────┴──────┐  ┌──────────────┐
   │ StripeScan  │              │ImageLoader   │  │Web Worker    │
   │ (gray code) │◄─────────────│  + Settings  │  │(brightest    │
   └─────────────┘              └──────────────┘  │ point)       │
                                                  └──────────────┘
```

## Camera System

The camera system uses a polymorphic interface pattern, separating image capture from settings control:

```
CameraInterface
├── ImageLoader (captures images)
│   ├── DummyImageLoader (base / no-op)
│   ├── IPImageLoader (IP/GeniCam via laser-server)
│   ├── USBImageLoader (UVC webcams via getUserMedia)
│   ├── AxisLoader (Axis IP cameras)
│   ├── WebRTCImageLoader (WebRTC streams)
│   └── VideoServer (WebRTC signaling)
│
└── SettingsInterface (controls exposure, gain, etc.)
    ├── DummyCameraSettingsInterface (base / no-op)
    ├── IPLaserServerSettings (laser-server REST API)
    ├── UVCServerSettings (uvcc-webserver on Mac)
    ├── BrowserUSBSettings (browser-native MediaStreamTrack)
    └── AxisSettings (Axis HTTP API)
```

`CameraInterface.detectCamera()` auto-selects the correct loader and settings driver based on the camera label and platform.

## Gray Code Scanning

The scanning pipeline establishes a mapping between camera pixels and projector pixels using Gray code patterns:

1. **StripeScan** projects alternating black/white stripe patterns (Gray code) on the screen
2. For each pattern, the camera captures an image
3. By analyzing which stripes each camera pixel sees as bright or dark, a binary code is assigned
4. The binary code decodes to a projector coordinate
5. **CameraRaster** stores the mapping: camera pixel -> projector (x, y)
6. **ProjectorRaster** stores the reverse: projector cell -> camera (x, y)

### Scan Types

- **flatScan()** - Baseline scan of a flat surface
- **calibrationMoundScan(x0, y0, x1, y1)** - Scan with a known raised object (for height calibration)
- **moundScan()** - Scan the current surface to generate a height map

### 3D Height Estimation

1. `flatScan()` establishes the baseline correspondence
2. `calibrationMoundScan()` learns how height differences shift the correspondence
3. `EpinodeEstimate` calculates the angle of displacement
4. `DiffRaster` computes per-cell height differences between flat and mound scans
5. Heights are normalized and can be visualized or exported

## Laser Detection Pipeline

```
Camera Frame
    │
    ▼
Background Subtraction (optional)
    │
    ▼
findBrightestPoint (Web Worker)
    │
    ▼
Brightness Threshold Check
    │
    ▼
Kalman Filter (smoothing)
    │
    ▼
State Machine (on/off/move)
    │
    ▼
Correspondence Lookup (camera → projector)
    │
    ▼
Screen Coordinate Mapping
    │
    ▼
Synthetic DOM Events (mouseover, click, etc.)
```

1. **PointerDetectorClient** runs the detection loop, continuously grabbing camera frames
2. **findBrightestPoint** (in a web worker for performance) finds the brightest pixel
3. If brightness exceeds the threshold, a `laser-on` or `laser-move` event fires
4. **KalmanFilterCoordinates** smooths the position data
5. **InputSimulator** maps the camera position through the `CameraRaster` to projector coordinates
6. Projector coordinates are scaled to screen pixels
7. **SyntheticEvents** dispatches mouse events on the DOM element at that position

## Event System

All major classes extend `Evented`, a lightweight pub/sub system:

```js
// Subscribe
const unsub = obj.on('eventName', callback)

// Unsubscribe
unsub()
// or
obj.off('eventName', callback)

// Emit
obj.fire('eventName', data)
```

Events are dispatched asynchronously via `setTimeout`.

## Module System

- Pure ES6 modules with explicit imports/exports
- Single external dependency: `simple-statistics` (for ckmeans clustering in scan analysis)
- Web workers used for computationally intensive image processing
- All classes are browser-only (Canvas, ImageData, getUserMedia, etc.)

## External Server Dependencies

| Server | Port | Purpose |
|--------|------|---------|
| laser-server | 8080 | GeniCam/PointGrey camera control (Python) |
| uvcc-webserver | 3456 | USB camera settings on Mac (Node.js) |

These are only needed for specific camera types. USB webcams on Windows/Android work directly through the browser.