Photoacoustic mesoscopy visualises vascular architecture at high-resolution up to ~3 mm depth. Despite promise in preclinical and clinical imaging studies, with applications in oncology and dermatology, the accuracy and precision of photoacoustic mesoscopy is not well established. Here, we evaluate a commercial photoacoustic mesoscopy system for imaging vascular structures. Typical artefact types are first highlighted and limitations due to non-isotropic illumination and detection are evaluated with respect to rotation, angularity, and depth of the target. Then, using tailored phantoms and mouse models, we investigate system precision, showing coefficients of variation (COV) between repeated scans [short term (1 h): COV= 1.2%; long term (25 days): COV= 9.6%], from target repositioning (without: COV=1.2%, with: COV=4.1%), or from varying in vivo user experience (experienced: COV=15.9%, unexperienced: COV=20.2%). Our findings show robustness of the technique, but also underscore general challenges of limited-view photoacoustic systems in accurately imaging vessel-like structures, thereby guiding users when interpreting biologically-relevant information.