> For the complete documentation index, see [llms.txt](https://multiset.gitbook.io/multiset/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://multiset.gitbook.io/multiset/native-support/ios-swift-native/sample-views/arlocalizationview.md). # ARLocalizationView ## Overview The `ARLocalizationView` provides AR-based localization using the MultiSetSDK framework. This view supports both single-frame and multi-frame localization modes, capturing frames from the AR camera and sending them to the MultiSet backend for pose estimation. ## Description The AR localization view handles both localization modes: **Single-Frame Localization** is ideal for scenarios where: * Quick localization is needed * The environment has distinct visual features * Network bandwidth is limited * Lower accuracy is acceptable **Multi-Frame Localization** is ideal for scenarios where: * Higher accuracy is required * The environment has repetitive or sparse visual features * The user can move the device slightly during capture * More robust localization is needed The view handles: * AR session management using ARKit and RealityKit * Frame capture and processing (single or multi-frame) * Localization API requests * Pose calculation and gizmo positioning * Mesh visualization * Background localization * GPS hint integration *** ## How It Works ### 1. Initialization Flow ``` ARLocalizationView appears └─> setupCallbacks() └─> Configure SDK callbacks for results └─> startGpsUpdates() (if GPS enabled) └─> startAutoLocalizationIfNeeded() ``` ### 2. Localization Flow ``` User taps Capture / Auto-localization triggers └─> MultiSet.shared.localize() └─> SDK captures frame(s) └─> [Multi-frame: Loop capture with intervals] └─> Send to API └─> onLocalizationSuccess / onLocalizationFailure └─> Update gizmo position └─> Load mesh visualization ``` *** ## Key Features ### Auto-Localization When `MultiSetConfig.autoLocalize` is enabled, localization starts automatically after the AR session is ready (with a 1.5 second delay to ensure tracking stability). ### Background Localization When `MultiSetConfig.backgroundLocalization` is enabled, the SDK periodically sends localization requests to refine positioning at the interval specified by `bgLocalizationDurationSeconds`. ### Relocalization When `MultiSetConfig.relocalization` is enabled, automatic relocalization is triggered when AR tracking state becomes paused or stopped. ### GPS Hint When `MultiSetConfig.passGeoPose` is enabled, GPS coordinates are captured and sent as a hint to improve localization accuracy for large-scale maps. ### Mesh Visualization When `MultiSetConfig.meshVisualization` is enabled, a 3D mesh overlay is rendered after successful localization. ### Multi-Frame Capture For multi-frame mode, captures multiple frames (configured via `numberOfFrames`) with intervals between captures (configured via `frameCaptureIntervalMs`). Each frame includes: * Image data (JPEG compressed) * Camera position (X, Y, Z) * Camera rotation (quaternion) ### Visual Feedback During multi-frame capture, an animated phone icon guides users to move their device, ensuring diverse viewpoints for better localization. *** ## SDK Methods Used ### Localization Control | Method | Description | | ----------------------------------------- | ------------------------------------ | | `MultiSet.shared.localize()` | Triggers the localization process | | `MultiSet.shared.stopLocalization()` | Stops ongoing localization | | `MultiSet.shared.clearMesh()` | Removes mesh visualization | | `MultiSet.shared.setLocalizationMode(_:)` | Changes localization mode at runtime | ### GPS Control | Method | Description | | ----------------------------------- | --------------------------- | | `MultiSet.shared.startGpsUpdates()` | Starts GPS location updates | | `MultiSet.shared.stopGpsUpdates()` | Stops GPS location updates | ### State Properties | Property | Type | Description | | ----------------------------------- | ----------------- | ------------------------------------------------ | | `MultiSet.shared.isLocalizing` | `Bool` | Whether localization is in progress | | `MultiSet.shared.hasLocalized` | `Bool` | Whether localization has succeeded at least once | | `MultiSet.shared.isShowingOverlay` | `Bool` | Whether API loading overlay should be shown | | `MultiSet.shared.isCapturingFrames` | `Bool` | Whether frames are being captured | | `MultiSet.shared.config` | `MultiSetConfig?` | Current SDK configuration | *** ## State Management ### View State Properties | Property | Type | Description | | ----------------------- | ------ | ------------------------------------------ | | `isTrackingNormal` | `Bool` | Whether AR tracking state is normal | | `hasLocalized` | `Bool` | Whether localization has succeeded | | `isLocalizing` | `Bool` | Polled from SDK - localization in progress | | `isShowingOverlay` | `Bool` | Polled from SDK - showing API overlay | | `isCapturingFrames` | `Bool` | Polled from SDK - capturing frames | | `showCloseConfirmation` | `Bool` | Controls close confirmation alert | | `showToast` | `Bool` | Controls toast message visibility | ### State Polling The view polls SDK state every 0.1 seconds to update UI: ```swift let stateTimer = Timer.publish(every: 0.1, on: .main, in: .common).autoconnect() .onReceive(stateTimer) { _ in isLocalizing = MultiSet.shared.isLocalizing isShowingOverlay = MultiSet.shared.isShowingOverlay isCapturingFrames = MultiSet.shared.isCapturingFrames } ``` *** ## Callbacks ### onLocalizationSuccess Handles successful localization result: ```swift sdkDelegate.onLocalizationSuccess = { result in hasLocalized = true if MultiSet.shared.config?.showAlerts == true { showToastMessage("Localization successful", success: true) } } ``` ### onLocalizationFailure Handles localization failure: ```swift sdkDelegate.onLocalizationFailure = { error in if MultiSet.shared.config?.showAlerts == true { showToastMessage(error, success: false) } } ``` ### onTrackingChanged Handles AR tracking state changes: ```swift sdkDelegate.onTrackingChanged = { state in isTrackingNormal = (state == .tracking) } ``` ### onMeshLoaded Handles mesh loading completion: ```swift sdkDelegate.onMeshLoaded = { mapId in print("Mesh loaded for map: \(mapId)") } ``` *** ## Actions ### startLocalization() Manually triggers the localization process: ```swift private func startLocalization() { MultiSet.shared.localize() } ``` ### resetWorldOrigin() Resets the AR world origin and clears current localization: ```swift private func resetWorldOrigin() { MultiSet.shared.stopLocalization() MultiSet.shared.clearMesh() hasLocalized = false showToastMessage("World origin reset", success: true) // Restart auto-localization if enabled startAutoLocalizationIfNeeded() } ``` ### cleanup() Cleans up resources when view disappears: ```swift private func cleanup() { MultiSet.shared.stopLocalization() MultiSet.shared.stopGpsUpdates() } ``` *** ## Configuration The view reads configuration from `MultiSet.shared.config`: ```swift // Example configuration settings config.localizationMode = .multiFrame // or .singleFrame config.autoLocalize = true config.backgroundLocalization = true config.bgLocalizationDurationSeconds = 30.0 config.relocalization = true config.numberOfFrames = 4 // For multi-frame config.frameCaptureIntervalMs = 500 // For multi-frame config.confidenceCheck = true config.confidenceThreshold = 0.3 config.firstLocalizationUntilSuccess = true config.showAlerts = true config.meshVisualization = true config.passGeoPose = false config.geoCoordinatesInResponse = false config.imageQuality = 90 ``` *** ## Usage Example ### Basic Usage ```swift import SwiftUI import MultiSetSDK struct MyARView: View { @ObservedObject var sdkDelegate: MultiSetSDKDelegate let localizationMode: LocalizationMode var body: some View { ZStack { // AR View from SDK MultiSetARView() .ignoresSafeArea() // Capture button VStack { Spacer() Button("Localize") { MultiSet.shared.localize() } .padding() } } .onAppear { setupCallbacks() } } private func setupCallbacks() { sdkDelegate.onLocalizationSuccess = { result in print("Localized at: \(result.position)") } } } ``` ### Navigating from LandingView ```swift NavigationStack { // ... LandingView content ... } .navigationDestination(isPresented: $navigateToAR) { ARLocalizationView( sdkDelegate: sdkDelegate, localizationMode: selectedMode ) .navigationBarBackButtonHidden(true) } ``` *** ## Comparison: Single-Frame vs Multi-Frame | Feature | Single-Frame | Multi-Frame | | ----------------------- | ------------------ | -------------------- | | Frames captured | 1 | 4-6 (configurable) | | Capture time | Instant | \~2-3 seconds | | Accuracy | Good | Better | | Network usage | Lower | Higher | | Movement during capture | Not required | Recommended | | Best for | Quick localization | Precise positioning | | Visual feedback | Simple overlay | Animated phone guide | *** ## Lifecycle ### onAppear * Sets up SDK callbacks for results * Starts GPS updates if enabled * Triggers auto-localization if enabled ### onDisappear * Stops localization * Stops GPS updates * Cleans up resources ### Timer * Polls SDK state every 0.1 seconds for UI updates *** ## Overlay Components ### LocalizationOverlay Shows during frame capture with animated phone image: * Semi-transparent black background * Background AR illustration * Animated phone icon (moving left to right) * "Hold steady and scan your surroundings" instruction * Status text showing capture progress ### APILoadingOverlay Shows while waiting for server response: * Semi-transparent black background * Circular progress spinner * "Localizing..." text *** ## Related * [LandingView](/multiset/native-support/ios-swift-native/sample-views/landingview.md) * [MultiSetConfig](/multiset/native-support/ios-swift-native/api-reference/multisetconfig.md) --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://multiset.gitbook.io/multiset/native-support/ios-swift-native/sample-views/arlocalizationview.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.