Metabolism is a fundamental process of life. However, non-invasive measurement of local tissue metabolism is limited today by a deficiency in adequate tools for in vivo observations. We designed a multi-modular platform that explored the relation between local tissue oxygen consumption, determined by label-free optoacoustic measurements of hemoglobin, and concurrent indirect calorimetry obtained during metabolic activation of brown adipose tissue (BAT). By studying mice and humans, we show how video-rate handheld multi-spectral optoacoustic tomography (MSOT) in the 700-970 nm spectral range enables non-invasive imaging of BAT activation, consistent with positron emission tomography findings. Moreover, we observe BAT composition differences between healthy and diabetic tissues. The study consolidates hemoglobin as a principal label-free biomarker for longitudinal non-invasive imaging of BAT morphology and bioenergetics in situ. We also resolve water and fat components in volunteers, and contrast MSOT readouts with magnetic resonance imaging data.