> For the complete documentation index, see [llms.txt](https://multiset.gitbook.io/multiset/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://multiset.gitbook.io/multiset/fundamentals/rest-api-docs/map-query.md). # Map Query ### VPS Query API Query your map to get the device position with respect to Map local origin. The query API takes query image (encoded as base64 string) and other image metadata. {% hint style="warning" %} The maximum query image resolution is **1280** pixels in either width or height. {% endhint %} {% openapi src="/files/ajrrlVmhtUvgT9mQ0Qyq" path="/vps/map/query" method="post" %} [VPSquery.yaml](https://3163433004-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FokTDI7QVY04Zvb1pQ8Ry%2Fuploads%2Fgit-blob-eb05fe5c99624689fc2491803b890e308d19077f%2FVPSquery.yaml?alt=media) {% endopenapi %} #### Sample Response (JSON Body) ```json { "poseFound": true, "position": { "x": -5.89516855615433, "y": 1.225031596452081, "z": 2.2112895596804227 }, "rotation": { "x": -0.007873432249486393, "y": 0.8212519784928444, "z": 0.03204363652415735, "w": 0.5696107462509017 }, "confidence": 0.46875, "mapIds": ["67e12d4bff7ecf561f2f8a0c"], "mapCodes": ["MAP_RJFKKWQ1787J"], "responseTime": 2572 } ``` {% openapi src="/files/4THolmFT4ct9qovYeocP" path="/vps/map/query-form" method="post" %} [VPSqueryform.yaml](https://3163433004-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FokTDI7QVY04Zvb1pQ8Ry%2Fuploads%2Fgit-blob-5899f4d33a9109b79fcd901da3e8c30099c7769b%2FVPSqueryform.yaml?alt=media) {% endopenapi %} #### Sample Response (Form Data) ```json { "poseFound": true, "position": { "x": -2.8765866867008025, "y": 1.4018881843419664, "z": 7.677072632098843 }, "rotation": { "x": -0.0033476528966347096, "y": 0.6750939967230872, "z": -0.00310829178339555, "w": 0.7377175796541121 }, "confidence": 0.9175769612711023, "mapIds": ["67e12d4bff7ecf561f2f8a0c"], "mapCodes": ["MAP_RJFKKWQ1787J"], "responseTime": 2669 } ``` {% hint style="warning" %} **FormData Content-Type:** When using fetch or similar HTTP clients with FormData, do not manually set the Content-Type header. The browser automatically sets it to multipart/form-data with the required boundary string. Manually setting it will strip the boundary and cause a parse error. {% endhint %} #### Code Example (Python) ```python import json import requests BASE_URL = "https://api.multiset.ai/v1" TOKEN = "YOUR_M2M_TOKEN" with open("query.jpg", "rb") as f: response = requests.post( f"{BASE_URL}/vps/map/query-form", headers={"Authorization": f"Bearer {TOKEN}"}, data={ "mapCode": "MAP_RJFKKWQ1787J", "isRightHanded": "false", "width": "720", "height": "960", "fx": "670.4620971679688", "fy": "670.4620971679688", "px": "478.838623046875", "py": "365.346618652343", # Optional localization hints, all as strings "hintPosition": "-3.452,0.252,-0.531", "hintRadius": "15", "hintFloorHeight": "[1.5, 4.5]", }, files={"queryImage": ("query.jpg", f, "image/jpeg")}, ) print(response.status_code, response.json()) ``` ### Spatial Hint: hintPosition The `hintPosition` parameter restricts the localization search to a radius around a given point in the map, speeding up the query and improving accuracy in visually repetitive scenes. | Parameter | Type | Description | | -------------- | ----------- | ----------------------------------------------------------------------- | | `hintPosition` | `[x, y, z]` | Hint point in the map's **left-handed (LHS / Unity) coordinate system** | | `hintRadius` | `number` | Search radius in meters (default `25`, range `5`–`100`) | {% hint style="warning" %} **`hintPosition` is in LHS (Unity) coordinates**, the same coordinate system returned in the `position` field of a successful localization response (`isRightHanded: false`). A previous localization result's `position` can be passed directly as the hint. If your application works in right-handed space (e.g. ROS, Three.js), convert to LHS before sending. {% endhint %} On form-data endpoints, pass `hintPosition` as a JSON-encoded string, e.g. `"[2.5, 0.1, 8.0]"`. See [Pose Prior : HintPosition](/multiset/fundamentals/localization/pose-prior-hintposition.md) for the full guide. ### Floor-Level Search: hintFloorHeight The `hintFloorHeight` parameter limits the search to a specific vertical band, enabling floor-level localization in multi-floor buildings. | Parameter | Type | Description | | ----------------- | ---------------- | ---------------------------------------- | | `hintFloorHeight` | `[y_min, y_max]` | Vertical band in map-local Y coordinates | **Rules:** * Both values are in map-local coordinates (or global MapSet coordinates for MapSet queries) * Order does not matter: `[-3, -1]` and `[-1, -3]` both select images with y ∈ \[-3, -1] * Can be combined with `hintPosition`: spatial filter runs first, height filter applies on the result **Examples:** ```json // Floor 2 of a building where floor is at y=3.0, ceiling at y=6.0 "hintFloorHeight": [3.0, 6.0] // Basement (negative Y) "hintFloorHeight": [-3.0, -0.5] // Form-data endpoints, pass as JSON string "hintFloorHeight": "[1.5, 4.5]" ``` ### Query Mode: queryMode The `queryMode` parameter selects which localization engine runs your query, letting you trade latency for accuracy. The request and response format stay exactly the same on both engines. | Value | Engine | Description | | ------- | ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `vps-1` | Standard (default) | The fast standard single-image localization engine, roughly 2 seconds. Best for real-time, interactive localization. | | `vps-2` | Deep Search | A deep-search engine with up to **15% higher recall** and improved accuracy, at higher latency (around 3 to 4 seconds). Best for offline or background workflows and challenging, visually repetitive scenes. | **Rules:** * Optional. When omitted, the query runs on `vps-1`, so existing integrations are unaffected. * Supported on both single-image endpoints: `/vps/map/query` (JSON) and `/vps/map/query-form` (form-data). It is ignored by the multi-image query endpoint. * Can be combined with all localization parameters (`hintPosition`, `hintRadius`, `hintFloorHeight`, `geoHint`, `hintMapCodes`). * Works with Maps, MapSets, and versioned maps. **Examples:** ```json // JSON body: run this query on the Deep Search engine "queryMode": "vps-2" // Form-data endpoints: pass as a plain string field "queryMode": "vps-2" ``` {% hint style="info" %} Use `vps-2` when accuracy matters more than latency, for example when placing persistent content or localizing in repetitive corridors. For high-frequency relocalization during an AR session, `vps-1` remains the recommended default. You can also try both engines on your own datasets from the Developer Portal simulation viewer. See [Query Mode](/multiset/fundamentals/localization/query-mode.md) for guidance on choosing between the two. {% endhint %} ### VPS Multi Image Query API Multi-image query API requires a minimum of 4 images in each request. Up to 6 images can be passed. Higher image counts increase localization robustness and accuracy but may compromise latency. {% hint style="info" %} **About image#\_data properties:** The `image1_data`, `image2_data`, etc. fields contain the local SLAM tracking data (position and rotation) for when each image was captured. This is the device pose in the local coordinate system from ARKit (iOS) or ARCore (Android) at the moment the image was taken. {% endhint %} {% openapi src="/files/oOKFI0YnlAkR5K578gJk" path="/vps/map/multi-image-query" method="post" %} [vps-maps-api.yaml](https://3163433004-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FokTDI7QVY04Zvb1pQ8Ry%2Fuploads%2Fgit-blob-055cacd941f1f381e5dd37cfa528e2707ede54e1%2Fvps-maps-api.yaml?alt=media) {% endopenapi %} #### Sample Response (Multi Image) ```json { "poseFound": true, "estimatedPose": { "position": { "x": 3.9023228363353613, "y": 2.2199969114542415, "z": 7.684019738005462 }, "rotation": { "x": 0.0008889080606250241, "y": 0.7454695784085924, "z": -0.022682644709609446, "w": 0.6661529967948453 } }, "trackingPose": { "position": { "x": 0.0022450201213359833, "y": 2.471872329711914, "z": -10.018059730529785 }, "rotation": { "x": 0.030655404552817345, "y": 0.14812710881233215, "z": -0.00305502163246274, "w": -0.9884883761405945 } }, "imageId": "image4", "mapIds": ["67e12d4bff7ecf561f2f8a0c"], "confidence": 0.34274043817304123, "frames": [ { "imageId": "image1", "poseFound": true, "position": { "x": 3.8404791227185512, "y": 2.2358714094377391, "z": 7.5311847290014732 }, "rotation": { "x": 0.0104227310558921, "y": 0.7218904471209736, "z": -0.0248105549134402, "w": 0.6914338805274115 }, "confidence": 0.29881204871834762 }, { "imageId": "image2", "poseFound": true, "position": { "x": 3.8697402118840365, "y": 2.2281455037103951, "z": 7.6015528841019182 }, "rotation": { "x": 0.0061140925713855, "y": 0.7331886201553348, "z": -0.0235510983247741, "w": 0.6795726114402537 }, "confidence": 0.31204558839107514 }, { "imageId": "image3", "poseFound": false }, { "imageId": "image4", "poseFound": true, "position": { "x": 3.9023228363353613, "y": 2.2199969114542415, "z": 7.684019738005462 }, "rotation": { "x": 0.0008889080606250241, "y": 0.7454695784085924, "z": -0.022682644709609446, "w": 0.6661529967948453 }, "confidence": 0.34274043817304123 } ], "mapCodes": ["MAP_RJFKKWQ1787J"], "responseTime": 4919 } ``` #### Per-image poses (`frames`) Alongside the single `estimatedPose`, the response carries a `frames` array with one entry per image you sent, in the order the images were numbered. Each entry reports whether that individual image localized on its own, and if it did, the 6-DoF pose computed from it. | Field | Type | Description | | ------------ | ------- | --------------------------------------------------------------------------------------------- | | `imageId` | string | Which image this entry describes (`image1` to `image6`), matching the file part you uploaded. | | `poseFound` | boolean | Whether this individual image produced a pose. | | `position` | object | `x`, `y`, `z` in map-local coordinates. Present only when `poseFound` is `true`. | | `rotation` | object | Quaternion `x`, `y`, `z`, `w`. Present only when `poseFound` is `true`. | | `confidence` | number | Confidence score for this image alone. May be absent on a frame that did not localize. | **How `frames` relates to the top-level pose:** * `estimatedPose` remains the answer to use. It is the consensus pose across all images and is more robust than any single frame. * The top-level `imageId` names the winning frame, so the entry with that `imageId` carries the same `position`, `rotation`, and `confidence` as `estimatedPose`. * Every pose in `frames` is in the same coordinate frame as `estimatedPose`: it follows the `isRightHanded` value you sent, and on a versioned map it is expressed in the base map's frame. * Individual frames can fail while the overall query succeeds. A frame with `"poseFound": false` has no `position` or `rotation`, which is normal when one image in the set is blurred, occluded, or pointed at a featureless surface. * `frames` is informational. It is useful for diagnosing capture quality, spotting outlier viewpoints, and understanding which frames drove the result, but you do not need it for standard localization. {% hint style="warning" %} **FormData Content-Type:** When using fetch or similar HTTP clients with FormData, do not manually set the Content-Type header. The browser automatically sets it to multipart/form-data with the required boundary string. Manually setting it will strip the boundary and cause a parse error. {% endhint %} #### Code Example (Python) Split the request in two parts: the image binaries go in `files`, and every other field, including each `imageN_data` tracking pose, goes in `data` as a string. ```python import json import requests from contextlib import ExitStack BASE_URL = "https://api.multiset.ai/v1" TOKEN = "YOUR_M2M_TOKEN" image_paths = ["image1.jpg", "image2.jpg", "image3.jpg", "image4.jpg"] # Local SLAM tracking pose (ARKit / ARCore) for each image, in the same order tracking_poses = [ {"x": -5.1772, "y": 0.2936, "z": -2.6439, "qx": -0.0185, "qy": 0.9949, "qz": -0.0691, "qw": 0.0703}, {"x": -4.8210, "y": 0.3011, "z": -2.1094, "qx": -0.0201, "qy": 0.9932, "qz": -0.0805, "qw": 0.0812}, {"x": -4.3945, "y": 0.2874, "z": -1.7723, "qx": -0.0173, "qy": 0.9951, "qz": -0.0664, "qw": 0.0699}, {"x": -3.9012, "y": 0.2952, "z": -1.3388, "qx": -0.0190, "qy": 0.9940, "qz": -0.0731, "qw": 0.0764}, ] # Text fields, all sent as strings text_fields = { "mapCode": "MAP_RJFKKWQ1787J", # or "mapSetCode", exactly one of the two "isRightHanded": "false", "width": "720", "height": "960", "fx": "670.4620971679688", "fy": "670.4620971679688", "px": "478.838623046875", "py": "365.346618652343", # Optional localization hints "hintPosition": "-3.452,0.252,-0.531", } # Each imageN file part needs a matching imageN_data text part for index, pose in enumerate(tracking_poses, start=1): text_fields[f"image{index}_data"] = json.dumps(pose) with ExitStack() as stack: files = [ ( f"image{index}", (f"image{index}.jpg", stack.enter_context(open(path, "rb")), "image/jpeg"), ) for index, path in enumerate(image_paths, start=1) ] response = requests.post( f"{BASE_URL}/vps/map/multi-image-query", headers={"Authorization": f"Bearer {TOKEN}"}, data=text_fields, files=files, ) print(response.status_code, response.json()) ``` {% hint style="info" %} **Rules for the multipart body:** * File parts must be named `image1` to `image6`, and each one needs a matching `imageN_data` text part with the same number. A missing `imageN_data` returns `400` with `Missing metadata for imageN`. * Each `imageN_data` value is a JSON-encoded string containing `x`, `y`, `z`, `qx`, `qy`, `qz` and `qw`. All seven keys are required. * Send between 4 and 6 images. Fewer returns `At least 4 image file is required!`, more returns `Maximum 6 image files allowed!`. * Pass exactly one of `mapCode` or `mapSetCode`. * Do not set the `Content-Type` header yourself. `requests` adds `multipart/form-data` with the boundary when you pass `files`. {% endhint %} --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://multiset.gitbook.io/multiset/fundamentals/rest-api-docs/map-query.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.