> For the complete documentation index, see [llms.txt](https://multiset.gitbook.io/multiset/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://multiset.gitbook.io/multiset/basics/localization.md).
# Localization
### What is Localization?
Localization is the process of determining a device's precise position and orientation (6-DoF pose) within a pre-scanned map. When a query image is sent to the VPS API, the system searches the map's database of reference frames for visually similar candidates, then computes an exact pose using feature matching and geometry.
By default the search covers the entire map. In large maps or multi-floor buildings this can be slow and prone to confusion between visually similar areas. **Localization parameters** let you provide prior knowledge (floor level, known position, GPS coordinates, map scope) to narrow the search before it begins, improving both speed and accuracy.
### Query Types
MultiSet VPS offers two ways to localize a device against a map: a **single frame query** and a **multi frame query**. Both send camera imagery to the same localization engine and return a 6-DoF pose; they differ in how many frames are sent, how long the request takes, and how robust the result is.
#### Single Frame Query
A single frame query sends **one camera frame** (a base64-encoded image plus its metadata) to the VPS. It is the fastest option, with a typical response time of around 2 seconds, and is ideal when you need quick, frequent pose updates or are localizing in well-textured, stable environments.
* **Endpoints:** `POST /vps/map/query` (JSON body) and `POST /vps/map/query-form` (multipart form-data)
* **Input:** 1 query image (max 1280 px on the longer side) + metadata
* **Response time:** \~2 seconds
* **Best for:** fast, repeated localization in stable, feature-rich scenes
#### Multi Frame Query
A multi frame query sends **4 to 6 camera frames** together with the local SLAM tracking pose (position and rotation from ARKit or ARCore) captured at the moment of each image. Combining multiple viewpoints makes the result more robust and accurate, especially in challenging conditions such as repetitive textures, partial occlusion, or minor scene changes. The trade-off is higher latency, with a typical response time of up to around 5 seconds.
* **Endpoint:** `POST /vps/map/multi-image-query` (multipart form-data)
* **Input:** 4 to 6 query images + per-frame SLAM tracking pose
* **Response time:** up to \~5 seconds
* **Best for:** robust localization in difficult or visually repetitive environments
#### Choosing a Query Type
| | Single Frame Query | Multi Frame Query |
| --------------------- | --------------------------------------- | ---------------------------------- |
| Images per request | 1 | 4 to 6 |
| Endpoint | `/vps/map/query`, `/vps/map/query-form` | `/vps/map/multi-image-query` |
| Typical response time | \~2 seconds | up to \~5 seconds |
| Robustness | Good in stable scenes | Higher, handles challenging scenes |
| Best for | Fast, frequent updates | Accuracy in difficult conditions |
The localization parameters described below apply to both query types. See [Map Query](/multiset/basics/rest-api-docs/map-query.md) for full request and response details.
### Localization Parameters
#### hintPosition
A 3D coordinate `[x, y, z]` in map-local space (left-handed / Unity convention) that tells the server approximately where the device is. The search is limited to frames within a radius of that point (default 25 m, configurable via `hintRadius`).
| Format | Example |
| ------------------------------- | ----------------------------------- |
| Array of 3 numbers (JSON body) | `"hintPosition": [2.5, 0.1, 8.0]` |
| JSON-encoded string (form-data) | `"hintPosition": "[2.5, 0.1, 8.0]"` |
Coordinates use the same LHS coordinate system as the localization response (when `isRightHanded: false`, the default). You can pass a previous localization result's `position` directly as the hint for the next query.
Use when you have approximate position from BLE, UWB, WiFi fingerprinting, or a previous localization result. For GPS-georeferenced maps, use `geoHint` instead.
#### hintMapCodes *(MapSet only)*
A list of map codes that limits the search to specific maps within a MapSet. Useful in multi-floor buildings where floor-level context is already known (e.g. from a floor selector, QR code scan, or BLE beacon).
| Format | Example |
| ------------------------------- | ---------------------------------------------------------- |
| Array of strings (JSON body) | `"hintMapCodes": ["MAP_RJFKKWQ1787J", "MAP_XPQR3456ABCD"]` |
| JSON-encoded string (form-data) | `"hintMapCodes": "[\"MAP_RJFKKWQ1787J\"]"` |
Only valid when querying against a `mapSetCode`. Ignored for single-map queries.
#### hintFloorHeight
A vertical band `[y_min, y_max]` in map-local Y coordinates. Only frames whose camera position falls within this Y range are considered. Useful for floor-level filtering in multi-floor spaces without needing per-floor map codes.
| Format | Example |
| ------------------------------- | --------------------------------- |
| Array of 2 numbers (JSON body) | `"hintFloorHeight": [3.0, 6.0]` |
| JSON-encoded string (form-data) | `"hintFloorHeight": "[3.0, 6.0]"` |
Order does not matter — `[-3, -1]` and `[-1, -3]` both select y ∈ \[-3, -1]. See [HintFloorHeight](/multiset/basics/localization/hint-floor-height.md) for full details.
#### geoHint
The device's GPS coordinates `[latitude, longitude, altitude]` passed as a comma-separated string. For georeferenced maps, the server converts these to map-local coordinates and applies the same spatial radius filtering as `hintPosition`. Requires the map to be georeferenced in the MultiSet Developer Portal.
| Format | Example |
| ---------------------- | --------------------------------------- |
| Comma-separated string | `"geoHint": "37.4219983,-122.084,10.5"` |
Works for both single Maps and MapSets. Combine with `convertToGeoCoordinates: "true"` to receive the pose result back in GPS coordinates. See [GeoHint in Localization](/multiset/basics/localization/geohint-in-localization.md) for full details.
#### hintRadius
Controls the search radius around `hintPosition` or `geoHint`. Default is 25 m. Smaller values are faster but require a more accurate prior; larger values are more forgiving.
| Format | Range | Default |
| -------------------------- | ------- | ------- |
| Number (JSON body) | 1–100 m | 25 m |
| Numeric string (form-data) | `"25"` | 25 m |
Only applies when `hintPosition` or `geoHint` is provided. Has no effect otherwise.
#### use2DFiltering
When `true`, the spatial radius filter ignores the Y-axis (altitude) and uses only horizontal distance (X and Z). Useful for tall buildings or outdoor environments where GPS altitude is unreliable.
| Format | Default |
| --------------------------------------- | --------- |
| Boolean (JSON body) | `false` |
| String `"true"` / `"false"` (form-data) | `"false"` |
Only applies when `geoHint` is provided. Works for both Maps and MapSets. Has no effect with `hintPosition` alone.
### How Parameters Combine
Parameters are applied in sequence — each one narrows the candidate set further:
```
All map frames
→ hintMapCodes (keep only frames from these maps)
→ hintPosition / geoHint + hintRadius (keep frames within radius)
→ use2DFiltering (if set, ignore Y in radius check)
→ hintFloorHeight (keep frames within vertical band)
→ visual matching (match the query image against the remaining candidates)
```
Using more hints together reduces the search space more aggressively and is particularly effective in large, multi-floor MapSets.
***
### Quick Reference
| Parameter | Type | Applies to | Purpose |
| ----------------- | ---------------- | ------------ | ---------------------------------------- |
| `hintPosition` | `[x, y, z]` | Map / MapSet | Spatial radius filter from a known point |
| `hintMapCodes` | `[string, ...]` | MapSet only | Limit search to specific maps |
| `hintFloorHeight` | `[y_min, y_max]` | Map / MapSet | Vertical band filter |
| `geoHint` | `"lat,lon,alt"` | Map / MapSet | GPS-based spatial filter |
| `hintRadius` | number (1–100) | Map / MapSet | Radius for hintPosition / geoHint |
| `use2DFiltering` | boolean | Map / MapSet | Ignore Y-axis in geoHint radius check |
---
# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.
## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.
Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:
```
GET https://multiset.gitbook.io/multiset/basics/localization.md?ask=&goal=
```
`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.
Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.