Mashups are built by combining building blocks, which are commonly referred to as mashup components. These components are characterized by a high level of heterogeneity in terms of technologies, access methods, and the behavior they may exhibit within a mashup. Abstracting away this heterogeneity is the mission of the so-called mashup tools aiming at automating or semi-automating mashup development to serve non-programmers. The challenge is to ensure this abstraction mechanism does not limit the support for heterogeneous mashup components. In this paper, we propose a novel evaluation framework that can be applied to assess the degree to which a given mashup tool addresses this challenge. The evaluation framework can serve as a benchmark for future improved design of mashup tools with respect to heterogeneous mashup components support. In order to demonstrate the applicability of the framework, we also apply it to evaluate some existing tools.