PixelScope: Turning the iPhone Camera into an Intelligent Display Analysis System

A display is harder to diagnose than it looks.

A professional display can look acceptable from one chair and defective from another. Room light, viewing angle, the bezel around the active image, and the observer's own visual memory all change the result. A handheld camera adds another layer: autofocus, auto-exposure, and auto white balance will happily rewrite the scene as the test color changes.

PixelScope exists because those variables are not side issues. They are the measurement. The product is an iPhone-based display analysis system: it presents known test fields, keeps a geometric hold on the active image, locks supported camera parameters, collects many observations instead of one snapshot, and writes a report that can be reviewed later.

This note follows the local Swift implementation reviewed on September 9, 2026. Every code block is a short, contiguous excerpt from that source, with its original file, line numbers, and source text. The excerpts are portions of the app, not standalone sample programs.

Workflow Preview

PixelScope in action

The recording below shows a PixelScope scan in operation. Use the player controls to pause at pattern transitions. The implementation behind those transitions is what the rest of this note explains.

Open the original MP4 recording

Known fields, not guessed colors

PixelScope does not ask the camera to interpret an unknown picture on the wall. It drives the display under test with controlled 4K measurement rasters — White, Black, Red, Green, and Blue at 3840×2160 — and then asks a much narrower question: what did the locked camera record from that known field?

White comes first for a reason. It is the acquisition reference: a bright, full-area field on which the session can stabilize, lock supported camera parameters, and establish geometry before the darker and saturated fields begin. The measurement sequence then visits White, Black, Red, Green, and Blue in that order. Each entry pairs a settling state with a capture state so the transition is explicit in the workflow.

50            let sequence: [(ScanAnalysisPattern, ScanPhase, ScanPhase)] = [51                (.white, .whiteStabilizing, .capturingWhite),52                (.black, .blackSettling, .capturingBlack),53                (.red, .redSettling, .capturingRed),54                (.green, .greenSettling, .capturingGreen),55                (.blue, .blueSettling, .capturingBlue)56            ]

The session uses the same multiplexed acquisition routine for all five fields. It seals each collection of frames before moving on. Analysis, master-image writing, and report construction happen after the five collections are complete. Capturing first keeps the heavier work out of the timed pattern sequence.

A consistent camera reference

If focus, exposure, and white balance were allowed to hunt as the screen changed from white to black to red, the report would mix display behavior with camera behavior. PixelScope therefore locks the supported AE, AF, and AWB settings after the initial white stabilization interval, then holds that reference across the rest of the scan.

19        if device.isFocusModeSupported(.locked) {20            device.focusMode = .locked21        }22
23        let duration = device.exposureDuration24        let iso = min(max(device.iso, device.activeFormat.minISO), device.activeFormat.maxISO)25        if device.isExposureModeSupported(.custom) {26            device.setExposureModeCustom(duration: duration, iso: iso, completionHandler: nil)27        } else if device.isExposureModeSupported(.locked) {28            device.exposureMode = .locked29        }30
31        let gains = clampedWhiteBalanceGains(device.deviceWhiteBalanceGains, on: device)32        if device.isWhiteBalanceModeSupported(.locked) {33            device.setWhiteBalanceModeLocked(with: gains, completionHandler: nil)34        }

Custom exposure is used where the device supports it; otherwise the controller falls back to exposure lock. White-balance gains are clamped to the device range before lock. Focus position, exposure duration, ISO, gains, and camera modes are stored with the scan metadata. After the session, supported continuous automatic modes are restored.

One snapshot is not a measurement.

A single still can be sharp, blurred, slightly moved, or caught during a camera adjustment. PixelScope treats dwell time as a collection window, not as the exposure of one frame. During each pattern-specific dwell and capture phase, the app copies frames into its own memory, attaches the current pattern, tracking phase, geometry, and camera settings, and later decides which frames are fit for analysis.

243            let duration = phase.duration(in: timing)244            let started = Date()245            while Date().timeIntervalSince(started) < duration {246                try Task.checkCancellation()247                guard isGuideReady else { throw ScanWorkflowError.trackingLost }248                publishMeasurementContext(acquisition, phase: phase, pattern: pattern)249                let elapsed = Date().timeIntervalSince(sequenceStarted)250                patternCaptureProgressText = String(251                    format: "%@ %.1fs / %.1fs",252                    pattern.displayTitle,253                    min(elapsed, total),254                    total255                )256                try await Task.sleep(for: .milliseconds(50))257            }

The reviewed source configures distinct windows for the shared multiplex phases. Those values describe the implementation that was inspected. They are not a published acquisition specification, and they can change as the app develops.

175    var blackPhaseB0: TimeInterval = 0.20176    var blackPhaseCorner: TimeInterval = 0.40177    var blackPhaseB5: TimeInterval = 0.20

A timed window does not guarantee a fixed number of usable frames. The pipeline records observed, accepted, and rejected counts. Analysis can then qualify its findings from the actual acquisition rather than from elapsed time alone. That is why controlled acquisition matters: the report can say not only what was measured, but how much evidence supported the measurement.

Green crosses are engineering references, not decoration.

Handheld capture moves. The display does not always fill the frame, and the active image is not the same thing as the plastic or metal around it. PixelScope therefore places tracking marks at the corners of the generated field: a green cross with an inward-facing L-shaped companion. Those marks exist so the software can keep a quadrilateral on the active image while the operator or the phone moves.

A marker that helps geometry also contaminates photometry. The cross is extra light and extra structure on the very region being measured. PixelScope resolves that conflict in time. The field cycles through six visibility phases: all corners on, one corner off at a time, then all corners off.

80enum MultiplexTrackingPhase: String, Codable, CaseIterable {81    case p0AllOn82    case p1TopLeftOff83    case p2TopRightOff84    case p3BottomRightOff85    case p4BottomLeftOff86    case p5AllOff

The one-corner-off phases proceed clockwise from the top-left. While one corner is uncovered for sampling, the other three remain visible so tracking can continue. Visible markers are excluded from photometric sampling. The exclusion covers the cross and the L together, with additional margin for detector uncertainty.

221        let safety = extraMargin + detectorUncertainty222        let pair = CornerFiducialGeometry.pairFootprint(for: corner, tuning: tuning)223            .insetBy(dx: -safety, dy: -safety)224        return legacy.union(pair)

Black is the geometry problem that looking cannot solve.

On a black test field, dark display pixels, a black bezel, and a dim room can merge into one silhouette. A human observer often cannot say where the active image ends. A camera has the same difficulty, and it is worse if the tracking marks are also off: there is then no bright corner reference left to observe.

PixelScope does not pretend to rediscover four corners from an unmarked black field. Live cross tracking runs through the phases that still show markers. The final all-off phase inherits the last trusted display quadrilateral. Black uses the same multiplex engine as White and RGB, but it keeps its own B0–B5 report keys so the documentation can describe that field in the language already used for Black.

111    var contributesToMeasurement: Bool { true }112    var usesLiveCrossTracking: Bool { self != .p5AllOff }113    var inheritsTrustedQuad: Bool { self == .p5AllOff }114
115    func reportKey(for pattern: ScanAnalysisPattern) -> String {116        if pattern == .black {117            return blackPhase.reportKey118        }119        switch self {120        case .p0AllOn: return "P0"121        case .p1TopLeftOff: return "P1"122        case .p2TopRightOff: return "P2"123        case .p3BottomRightOff: return "P3"124        case .p4BottomLeftOff: return "P4"125        case .p5AllOff: return "P5"126        }127    }

Continuity is preserved without rewriting the evidence. Each trusted quad records how it was obtained: visible crosses, an estimated missing corner, a held hidden corner, or an inherited quadrilateral. Per-corner state is equally explicit. A visible observation is not the same thing as an estimated corner or an inherited one.

14enum FiducialTrackingSource: String, Codable, CaseIterable {15    case visibleCrosses16    case estimatedCorner17    case heldHiddenCorner18    case inheritedQuad

189enum FiducialCornerState: String, Equatable, Sendable {190    case visible191    case partial192    case estimated193    case inherited194    case intentionallyHidden195    case intentionallyHiddenHeld196    case temporarilyLost197    case propagated198    case reacquiring199    case requiredMissing200}

That distinction matters when the report is read later. Inherited geometry can keep the measurement aligned through the unmarked black interval. It is not presented as a fresh cross detection. Estimated corners can keep a moving camera in contact with the panel without claiming that every corner was directly seen in that frame.

Structured measurements, not a single impression

After the five collections are sealed, analysis runs on the master images. The engine computes channel means, luma variation, sample counts, and pattern-dependent anomaly statistics. The point is not to replace an engineer with a score. It is to put numbers next to the conditions that produced them.

43    static func analyze(store: TemporaryScanStore, metadata: ScanMetadata) throws -> AnalysisResults {44        var white = try? statistics(at: store.url(for: .white), field: .bright, quality: quality(.white, in: metadata))45        var black = try? statistics(at: store.url(for: .black), field: .black, quality: quality(.black, in: metadata))46        var red = try? statistics(at: store.url(for: .red), field: .bright, quality: quality(.red, in: metadata))47        var green = try? statistics(at: store.url(for: .green), field: .bright, quality: quality(.green, in: metadata))48        var blue = try? statistics(at: store.url(for: .blue), field: .bright, quality: quality(.blue, in: metadata))

Those measurements are organized around the questions a display inspection actually asks:

• Uniformity — how luma varies across the accepted samples of a known field.
• Black behavior — whether the black field stays dark, or whether leakage and structure appear once geometry is held.
• Color-channel behavior — independent Red, Green, and Blue response rather than a single mixed impression.
• Localized panel anomalies — clusters and persistent low-response samples that survive across frames and bright fields.
• Edge and corner behavior — abnormal rows and columns, where tracking marks would otherwise hide the very regions that matter.
• Backlight or panel irregularities — spatial structure in the master images that is worth investigating, not automatically a named defect.

Findings are qualified by acquisition quality. For Black, insufficient quality suppresses the anomaly counts rather than presenting empty statistics as a clean bill of health.

381        if blackRecord?.acquisitionQuality != .high {382            black?.leakageSampleCount = 0383            black?.clusterCount = 0384            black?.abnormalRowCount = 0385            black?.abnormalColumnCount = 0386        }

Zeroed counts must be read with that rule in mind: they can mean a finding was withheld. A low-response sample in a rectified camera image is evidence to inspect, not a one-to-one count of physical panel pixels.

Measurement first, interpretation second

PixelScope already captures monitor identity — brand, model, and manufacture year — and it already records measured facts from the scan. What it does not yet do is apply model-specific manufacturer tolerances or an aggregated PixelScope reference library. Those sources are explicitly marked unavailable. Missing references stay false; they are not filled with invented limits.

115    static func current(hasMeasuredFacts: Bool) -> ReportEvidenceSources {116        ReportEvidenceSources(117            measuredFactsAvailable: hasMeasuredFacts,118            verifiedManufacturerReferenceAvailable: false,119            aggregatedPixelScopeReferenceAvailable: false120        )121    }

The planned PixelScope AI Reference Intelligence layer sits after that measurement chain. The report pipeline already reserves a later interpretation stage. In the reviewed source, that stage advances progress and then builds the report from measured analysis. It is a place in the workflow, not a shipping intelligence engine.

278enum ReportProcessingPhase: Int, CaseIterable {279    case preparingCaptures280    case imagePreprocessing281    case displayAnalysis282    case aiInterpretation283    case buildingReportModel284    case renderingReport285    case savingReport286    case finalizing

The intended layer would combine PixelScope measured evidence with context the measurement alone cannot supply: monitor brand, model, manufacture year, display architecture where known, manufacturer technical documentation, credible engineering references, and peer-reviewed or technical literature where it applies. Accumulated diagnostic context from prior PixelScope work would enter only as identified reference material, not as anonymous authority.

The design constraint is the same as the current report model: measurement comes first. AI-assisted interpretation comes second, and only where the supporting references exist. When they do not, the system should disclose uncertainty rather than fabricate model-specific knowledge. A confident sentence that cannot be sourced is worse than a measured result with an empty reference field.

A scan should leave documentation behind.

PixelScope turns a diagnostic session into a record. One report model feeds the in-app preview and the PDF renderer, so the two views cannot drift apart. The saved package includes the report, the PDF, and pattern thumbnails. Temporary 4K masters are not kept in that archive.

17    static func render(_ report: ReportModel) throws -> Data {18        let presentation = ReportPresentation(report)19        let renderer = UIGraphicsPDFRenderer(bounds: pageRect)20        let data = renderer.pdfData { context in21            let layout = Layout(context: context)22            drawPage1(report, presentation, layout)23            drawPage2(presentation, layout)24            layout.beginPage(appendix: true)25            drawAppendix(presentation, layout)26            layout.drawFooter()27        }28        guard !data.isEmpty else { throw ScanWorkflowError.persistenceFailed }29        return data30    }

The document is built to be read by people who were not holding the phone:

• measurements from the five known fields
• observations qualified by accepted and rejected frames
• monitor identity when it was captured
• findings and analysis, with acquisition quality attached
• methodology, including tracking sources and camera locks
• an in-app report preview and a shareable PDF

Export uses the standard iOS share sheet, so the PDF can be sent through the destinations already on the phone. That matters in the field. The point of the workflow is not only to see something on the display. It is to leave a file that an engineering manager, an integrator, a facility, a service organization, or an equipment owner can reopen without repeating the scan.

Used that way, PixelScope is a portable instrument for people who already work with professional displays: field technicians checking a room, engineering managers reviewing a finding, integrators documenting a handover, and owners who need more than a photograph of a screen.

Implementation notes

Excerpts were verified against PixelScope source revision 3d37101f57cb. File paths in the captions are relative to the PixelScope project. They identify the reviewed source; the website does not require the Xcode project at runtime.

Some in-app report sentences still contain older dwell wording. This article describes controlled acquisition windows and pattern-specific dwell and capture phases. It does not treat those older sentences as a timing specification.