Description
In this work, the authors propose a model called Rex‑Omni — a 3-billion-parameter multimodal large language model (MLLM) — which reframes a wide range of visual-perception tasks (object detection, spatial grounding, key-pointing, OCR) as a “next-point prediction” problem. Instead of the traditional regression-based coordinate prediction widely used in detectors, Rex-Omni quantises spatial coordinates into discrete tokens and asks the model to predict the next token (or point) in a sequence, allowing detection and localization to be treated in the same token-prediction paradigm as language modelling. The paper argues that this unification leads to both stronger language-vision alignment and improved localization performance. They support their claims with evaluations on standard benchmarks (such as COCO) and show that Rex-Omni outperforms prior methods when given the same or less supervision. Beyond just detection, the design opens up a path toward more general “detect-anything” models where a single architecture handles many tasks.
