The standard way of construing representation in neuroscience thinks of neural activities as encoding information. The activity then contributes to the function of the system by communicating that information to its outputs. We argue that this way of thinking is in tension with a number of well-established facts about neural activities. In general, neural activities do not correspond to one particular environmental or experimental variable. They show mixed selectivity and mixed variance; they are multiplexed and context-sensitive. This suggests that no specific message is encoded in their activity. We articulate this critique and argue that it should motivate alternative ways of thinking about representation. Our preferred alternative is to think in terms of the latent structure of brain systems. These include attractors, subspaces, and manifolds, and are the result of learned synaptic and neuromodulatory connections between neurons in a population. On our view, representations are due to the interaction of the latent structure of a brain system with its environment. We explain the differences between this view and the encoding view, and flesh out our proposal by describing research into prefrontal physiology and engrams. We conclude by arguing that the latent-structure-based view posits genuine representations.