← simbionte.org
Published work

Research

Work on evaluation, intention, and meaning in AI systems. Each entry states its own editorial status.

A single question, revisited in a spiral: the 2023 paper proposed replacing imitation-based evaluation with language acquisition; the next built intention-level metrics for what happens before generation; the third put the program to experimental test with grounded word learning in artificial agents; the most recent checks that learner's visual distances against adult human word-extension data.

  1. Nov 2026
    arxiv:2611.14116
    Preprint · arXiv v1 · cs.CL · 10 Nov 2026
    Author: Patricio M. Vera

    Visual Similarity in Adult Word Extension and Artificial Word Learning

    When two objects share a name, how much does visual similarity explain, and what do the teaching examples add? A reanalysis of trial-level data from a large adult word-extension study (Lewis & Frank, 2018; 520 people) measures each test image's distance to the taught examples in the space of a frozen DINOv2 encoder: greater distance goes with less extension (log odds −3.23 per SD), and example composition matters as well. In an artificial learner built on the same encoder, words joining two categories are learned less accurately the farther apart the categories are (84.7%, 65.4%, 53.0% naming accuracy for near, mid, and far pairs). The encoder's distances track human similarity measures (ρ = .501–.795), yet that alignment does not yield conclusive evidence that human-derived distances predict learner difficulty.

    BibTeX
    @misc{vera2026visual,
      title={Visual Similarity in Adult Word Extension and Artificial Word Learning},
      author={Vera, Patricio M.},
      year={2026},
      eprint={2611.14116},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
    }
  2. Jun 2026
    arxiv:2606.22207
    Preprint · arXiv
    Author: Patricio M. Vera

    Lexical Consensus: Grounded Word Learning and Shared Meaning in Artificial Agents

    AI systems are typically evaluated through task performance and imitation, leaving open whether an agent can acquire, stabilize, and use new lexical meanings from grounded experience. This paper introduces Lexical Consensus, an experimental framework for grounded word learning over frozen DINOv2 visual embeddings using Carroll-style nonce words. The central finding is a robust perceptual-coherence gradient: acquisition success is governed by perceptual distance rather than semantic relatedness (partial R² = 0.245 vs 0.002), with bidirectional naming/retrieval tests exposing a memory-fidelity dimension separate from naming accuracy.

    BibTeX
    @misc{vera2026lexical,
      title={Lexical Consensus: Grounded Word Learning and Shared Meaning in Artificial Agents},
      author={Vera, Patricio M.},
      year={2026},
      eprint={2606.22207},
      archivePrefix={arXiv}
    }
  3. Jan 2026
    arxiv:2601.01011
    Preprint · arXiv
    Author: Patricio M. Vera

    Intention Collapse: Intention-Level Metrics for Reasoning in Language Models

    Every act of language generation compresses a rich internal state into a single token sequence. This paper formalizes that many-to-one projection, defines three model-agnostic metrics — intention entropy, effective dimensionality, and latent knowledge recoverability — and proposes an empirical agenda for studying how inference-time computation shapes internal intentions before verbalization.

    BibTeX
    @misc{vera2026intention,
      title={Intention Collapse: Intention-Level Metrics for Reasoning in Language Models},
      author={Vera, Patricio M.},
      year={2026},
      eprint={2601.01011},
      archivePrefix={arXiv}
    }
  4. Sep 2023
    arxiv:2309.11981
    Preprint · arXiv
    Authors: Patricio M. Vera, Pedro Moya, Lisa Barraza

    Rethinking the Evaluating Framework for Natural Language Understanding in AI Systems

    As machine cognitive evaluation has reached the stage of imitation, the next frontier is genuine language acquisition and understanding. This paper proposes a paradigm shift from the Turing Test toward an interdisciplinary framework grounded in language acquisition, taking inspiration from recent advances in large language models.

    BibTeX
    @misc{vera2023rethinking,
      title={Rethinking the Evaluating Framework for Natural Language Understanding in AI Systems},
      author={Vera, Patricio M. and Moya, Pedro and Barraza, Lisa},
      year={2023},
      eprint={2309.11981},
      archivePrefix={arXiv}
    }