{"id":13795,"date":"2026-05-07T11:52:57","date_gmt":"2026-05-07T11:52:57","guid":{"rendered":"https:\/\/www.gvam.es\/?p=13795"},"modified":"2026-05-07T11:54:32","modified_gmt":"2026-05-07T11:54:32","slug":"how-we-know-it-works-reevaluates-approach-to-pilot-evaluation","status":"publish","type":"post","link":"https:\/\/www.gvam.es\/en\/how-we-know-it-works-reevaluates-approach-to-pilot-evaluation\/","title":{"rendered":"How We Know It Works: REEVALUATE&#8217;s Approach to Pilot Evaluation"},"content":{"rendered":"<p><strong>The REEVALUATE Project at a Glance<\/strong><\/p>\n<p>REEVALUATE is a Horizon Europe Research and Innovation project (Grant Agreement No. 101132389) with a bold mission: to provide a holistic solution to the challenges of Cultural Heritage (CH) digitisation management. The project develops a <strong>modular framework<\/strong> that covers the entire life cycle of a digitised artefact \u2014 from prioritisation and contextualisation to storage, collaboration and creative reuse \u2014 through a set of powerful technological enablers. AI techniques, smart contracts, CH object tokenization, and standardised semantic representations are all part of the toolkit the consortium is building for the European CH sector.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-13796\" src=\"https:\/\/www.gvam.es\/wp-content\/uploads\/2026\/05\/ai-girl-vr-museum-e1709645883874-700x400.png\" alt=\"\" width=\"514\" height=\"294\" srcset=\"https:\/\/www.gvam.es\/wp-content\/uploads\/2026\/05\/ai-girl-vr-museum-e1709645883874-700x400.png 700w, https:\/\/www.gvam.es\/wp-content\/uploads\/2026\/05\/ai-girl-vr-museum-e1709645883874-768x439.png 768w, https:\/\/www.gvam.es\/wp-content\/uploads\/2026\/05\/ai-girl-vr-museum-e1709645883874-120x69.png 120w, https:\/\/www.gvam.es\/wp-content\/uploads\/2026\/05\/ai-girl-vr-museum-e1709645883874.png 1344w\" sizes=\"(max-width: 514px) 100vw, 514px\" \/><\/p>\n<p>The framework is being validated in <strong>three real-life pilot scenarios<\/strong> that demonstrate its practical value across the Fashion &amp; Gaming, Advertising, and Tourism industries. One pilot, for example, is developing immersive virtual tours of the archaeological site of Aquileia, where digitised artefacts prioritised through public sensing are woven into interactive storytelling environments. REEVALUATE&#8217;s enablers are being made available, free of charge, for integration into the <strong>European Collaborative Cloud for Cultural Heritage (ECCCH)<\/strong>.<\/p>\n<p><strong>Putting Our Pilots to the Test<\/strong><\/p>\n<p>As our pilots evolve, a dedicated evaluation team (Task 4.6) is systematically measuring their outcomes through a <strong>mixed-methods framework<\/strong> that captures both what happened and why it mattered. Our evaluation tracks <strong>two complementary families of indicators<\/strong>:<\/p>\n<ul>\n<li><strong>Measured (direct, numeric) KPIs<\/strong> \u2014 such as number of participants, responses, users, collaborations, downloads, sales, artefacts, visitors, and interactions with content across pilot sites and social networks.<\/li>\n<li><strong>Constructed (indirect, qualitative and quantitative) KPIs<\/strong> \u2014 gathered through focus groups, stakeholder workshops, satisfaction surveys, and ongoing interaction with community members.<\/li>\n<\/ul>\n<p>This dual approach ensures that the pilots are not only counted, but truly understood.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-13800\" src=\"https:\/\/www.gvam.es\/wp-content\/uploads\/2026\/05\/Captura-de-pantalla-2026-05-07-a-las-13.44.38-700x403.png\" alt=\"\" width=\"492\" height=\"283\" \/><\/p>\n<p><strong>Built on Solid Theoretical Foundations<\/strong><\/p>\n<p>Our evaluation methodology does not start from scratch \u2014 it builds on two carefully selected bodies of scientific literature:<\/p>\n<p>The <a href=\"https:\/\/www.google.com\/url?sa=t&amp;source=web&amp;rct=j&amp;opi=89978449&amp;url=https:\/\/ec.europa.eu\/newsroom\/horizon2020\/document.cfm%3Fdoc_id%3D52017&amp;ved=2ahUKEwi9g5bg1v6TAxWQgv0HHSs_PE4QFnoECB0QAQ&amp;usg=AOvVaw2ejiMr9RlyWEQhYmgNIlEI\"><strong>SIMRA Evaluation Manual<\/strong><\/a> (Secco et al., 2020), developed within a previous Horizon 2020 project, provides a flexible yet comprehensive guide for assessing social innovation initiatives. Its structured approach covers scope, timing, stakeholder strategy, and methodological design, and is already used to support European Commission evaluations of social impact. Its strength lies in its adaptability to diverse target users and its ability to guide the selection of evaluation questions through a mixed qualitative-quantitative approach.<\/p>\n<p>Complementing this, the work of <a href=\"https:\/\/www.researchgate.net\/publication\/349181551_Application_of_a_Comprehensive_Methodology_for_the_Evaluation_of_Social_Innovations_in_Rural_Communities\"><strong>Baselice et al.<\/strong><\/a><strong> (2021)<\/strong> offers a real-world case study of adapting the SIMRA framework to an early-stage research project, demonstrating how an evaluation grid can be used to select the most relevant indicators from existing social innovation literature. Their conclusions on adaptation strategies, methodological robustness, and stakeholder interaction capabilities directly shaped REEVALUATE&#8217;s own approach.<\/p>\n<p>To assess the <strong>usability<\/strong> of REEVALUATE&#8217;s technological tools, the team adopted the internationally recognised <a href=\"https:\/\/en.wikipedia.org\/wiki\/System_usability_scale\"><strong>System Usability Scale (SUS)<\/strong><\/a> \u2014 a reliable and validated 10-item instrument widely used in technology evaluation.<\/p>\n<p><strong>Listening to Stakeholders: The REEVALUATE Survey<\/strong><\/p>\n<p>A core instrument in our evaluation toolkit is the <strong>REEVALUATE Satisfaction and Usability Survey<\/strong>, deployed via the European Commission&#8217;s EUSurvey platform and addressed to all pilot participants \u2014 from archaeologists and educators to citizens and IT developers. The survey captures:<\/p>\n<ul>\n<li><strong>Satisfaction levels<\/strong> with REEVALUATE&#8217;s results and activities (on a 1\u201310 scale)<\/li>\n<li><strong>Perceived empowerment<\/strong> and improvement in professional practices<\/li>\n<li><strong>Stakeholder collaboration patterns<\/strong> before, during, and beyond the pilots<\/li>\n<li><strong>Challenges encountered<\/strong> (technological, ethical, IPR-related, skills-related)<\/li>\n<li><strong>Usability perceptions<\/strong> of REEVALUATE&#8217;s tools using the SUS scale<\/li>\n<li><strong>Motivations and future intentions<\/strong> for using or reusing REEVALUATE outputs<\/li>\n<\/ul>\n<p>Participation is voluntary and GDPR-compliant, and results will feed long-term improvement loops and inform aggregated reporting to the European Commission.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>The REEVALUATE Project at a Glance REEVALUATE is a Horizon Europe Research and Innovation project (Grant Agreement No. 101132389) with a bold mission: to provide a holistic solution to the challenges of Cultural Heritage (CH) digitisation management. The project develops a modular framework that covers the entire life cycle of a digitised artefact \u2014 from [&hellip;]<\/p>\n","protected":false},"author":12,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[2],"tags":[],"acf":[],"_links":{"self":[{"href":"https:\/\/www.gvam.es\/en\/wp-json\/wp\/v2\/posts\/13795"}],"collection":[{"href":"https:\/\/www.gvam.es\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.gvam.es\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.gvam.es\/en\/wp-json\/wp\/v2\/users\/12"}],"replies":[{"embeddable":true,"href":"https:\/\/www.gvam.es\/en\/wp-json\/wp\/v2\/comments?post=13795"}],"version-history":[{"count":3,"href":"https:\/\/www.gvam.es\/en\/wp-json\/wp\/v2\/posts\/13795\/revisions"}],"predecessor-version":[{"id":13810,"href":"https:\/\/www.gvam.es\/en\/wp-json\/wp\/v2\/posts\/13795\/revisions\/13810"}],"wp:attachment":[{"href":"https:\/\/www.gvam.es\/en\/wp-json\/wp\/v2\/media?parent=13795"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.gvam.es\/en\/wp-json\/wp\/v2\/categories?post=13795"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.gvam.es\/en\/wp-json\/wp\/v2\/tags?post=13795"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}