Context: The emergence of the long-tail in the market of software applications is shifting the role of end-users from mere consumers to becoming developers of applications addressing their unique, personal, and transient needs. On the Web, a popular form of such applications is called mashup, built out of the lightweight composition of Web APIs (reusable software components delivered as a service through the Web). To enable end-users to build mashups, there is a key problem that must be overcome: End-users lack programming knowledge as well as the interest to learn how to master the complex set of Web technologies required to develop mashups. End-User Development (EUD) is an emerging research field dealing with this type of problems. Its main goal is to design tools and techniques facilitating the development of software applications by non-programmers.

Objective: The paper describes the design and evaluation of NaturalMash, an innovative EUD tool for mashups (a mashup tool). NaturalMash aims at enabling non-professional users without any knowledge of programming languages and skills to create feature-rich, interactive, and useful mashups.

Methods: The design of NaturalMash adopts a formative evaluation approach, and has completed three design and evaluation iterations. The formative evaluations utilize usability testing, think aloud protocol, questionnaires, observation, and unstructured interviews. Additionally, we compare the expressive power of naturalmash with the state-of-the-art mashup tools.

Results: The results from the formative evaluations helped us identify important usability problems. From an assessment point of view, the results were promising and sggested that the proposed tool has a short and gentle learning curve in a way that even non-programmers are able to rapidly build useful mashups. Also, the comparative evaluation results showed that NaturalMash offers a competitive level of expressive power compared with existing mashup tools targeting non-programmers.

Conclusion: As the evaluation results indicate, NaturalMash provides a high level of expressive power while it is still highly usable by non-programmers. These suggest that we have successfully achieved the objective of the proposed tool, distinguishing it from existing mashup tools that are either too limited or highly specialized for non-professional users.