The Hessian-based Frangi vesselness filter is commonly used to enhance vasculature in optoacoustic (photoacoustic) images, but its accuracy and limitations have never been rigorously assessed. Here we validate the ability of the filter to enhance vessel-like structures in phantoms, and we introduce an experimental approach that uses measurements before and after the administration of gold nanorods (AuNRs) to examine filter performance in vivo. We evaluate the influence of contrast, filter scales, angular tomographic coverage, out-of-plane signals and light fluence on image quality, and gain insight into the performance of the filter. We observe the generation of artifactual structures that can be misinterpreted as vessels and provide recommendations to ensure appropriate use of Frangi and other vesselness filters and avoid misinterpretation of post-processed optoacoustic images.