Research Article

Supporting Design Evaluation in Studio Practice: A Data-Informed Framework Integrating Peer Assessment and Ergonomic Analysis

Rafael Coelho, Josee Luis Simao and José Simoes*

¹ Esad. idea - Research in Deign and Art, Matosinhos, Portugal

Submission: May 18, 2025;Published:June 01, 2026

*Corresponding author:Jose Simoes, Esad. idea - Research in Deign and Art, Matosinhos, Portugal

How to cite this article:Rafael C, Josee Luis S, José S. Supporting Design Evaluation in Studio Practice: A Data-Informed Framework Integrating Peer Assessment and Ergonomic Analysis. Eng Technol Open Acc 2026; 7(1): 555702.DOI: 10.19080/ETOAJ.2026.07.555702

Abstract

This study examines how data-informed approaches can enhance design evaluation in studio-based learning. It proposes a framework combining peer assessment, multi-criteria decision analysis and statistical methods to evaluate design prototypes. The framework was applied to seventeen kettle handle designs created by undergraduate students, assessed using five criteria: ease of use, safety, comfort, robustness, and proportions. Methods included descriptive statistics, ANOVA, correlation analysis, clustering, and principal component analysis. Results indicate that students evaluated designs in a balanced, multidimensional way, considering both ergonomic and formal aspects. Strong links were found between proportions, comfort, and ease of use, emphasizing the importance of geometric coherence. Clustering and component analysis revealed performance groupings and key evaluation dimensions, while multi-criteria analysis enabled clear comparison and ranking. Overall, the study shows how quantitative tools can support structured, reflective, and systematic design evaluation in studio contexts.

Keywords:Design evaluation; Studio-based learning; Peer assessment; Ergonomic design; multi-criteria decision analysis; Data-informed design

Introduction

Ergonomic considerations play a central role in the design of handheld tools and consumer products, particularly those requiring precise grip control, wrist stabilisation, and repetitive manipulation. Foundational studies have shown that handle geometry, including grip diameter, cross-sectional shape, and surface contouring, significantly influences user comfort, biomechanical load, and functional performance [1,2]. More recent research confirms that handle dimensions, shape, and material properties affect grip force, muscle activity, wrist posture, and perceived comfort during manual tasks [3-5]. These factors directly influence fatigue, grip stability, and task efficiency, and are closely related to individual differences in hand anthropometry, grip strength, and manual dexterity [6,7].

Variations in handle diameter, surface texture, and spacing between gripping surfaces have also been shown to affect torque generation, usability, and perceived comfort [8]. Poorly designed handles can increase muscular effort and contribute to musculoskeletal strain, whereas ergonomically optimised geometries improve force transmission, performance, and safety.

In particular, research on handle sizing highlights the importance of aligning grip dimensions with anthropometric variability to maximise force output and reduce discomfort. The relationship between grip force and task performance further reinforces the role of ergonomic interventions in improving efficiency and reducing physical strain [9]. These insights support the view that ergonomic evaluation should be embedded throughout the design process rather than applied retrospectively. Integrating ergonomics in early design stages, through iterative prototyping, testing, and simulation, can enhance product performance and reduce physical strain [10]. Advances in digital modelling, anthropometric analysis, and rapid prototyping technologies have further expanded designers’ ability to anticipate ergonomic performance prior to fabrication, enabling more informed design decisions during early development phases.

Research on consumer products and hand tool design further underscores the importance of ergonomics in everyday interaction, highlighting how material properties, form, and structural characteristics shape usability, safety, and user experience [11-13]. These perspectives align with inclusive design approaches, which emphasise accommodating diverse user capabilities and reducing exclusion through design [14,15]. Together, this body of work positions handle design as a critical interface through which users experience comfort, control, and product quality. Additionally, physical conditions such as wrist or neck pain may directly influence grip strength and performance, further reinforcing the importance of ergonomically appropriate design solutions [16].

Methodology

Participants and studio context

The study was conducted with second-year undergraduate students enrolled in a Product Design programme at the Escola Superior de Arte e Design (ESAD) during the first semester of the 2025/2026 academic year. Within the studio-based course Models and Prototypes, students were tasked with designing, prototyping, and evaluating handles for an electric kettle. Seventeen students participated, each producing a full-scale prototype and contributing to a structured peer-evaluation process. The assignment drew on contemporary product references, including the Fellow Stagg Electric Kettle, Kutto by Samuel Drew, and handle designs by Muuto, to situate the project within current design language and practice (Figure 1).

Design brief and prototype development

The project required students to develop multiple design proposals for a kettle handle and to select one concept for further refinement through volumetric modelling and prototyping. All designs were developed in relation to a standardised kettle body (approximately 2 litres, maximum load 2.0 kg), provided to ensure consistency in scale and functional constraints (Figure 2). The development process included sketching, mock-ups, computer aided design modelling, and fabrication of physical prototypes. Materials varied depending on individual approaches, including foam, additive manufacturing, and mixed construction techniques. Each prototype incorporated a computer numerical control -cut cross-sectional core aligned with the standardised fixing points of the kettle body, ensuring comparability across designs. The resulting models were designed to be handled and assessed under consistent conditions.

Evaluation framework and data collection

All prototypes were evaluated through a structured peerassessment process in which each participant assessed all designs except their own. Evaluations were conducted anonymously in a controlled setting, where participants could physically handle and inspect each prototype. This process generated a complete dataset of 17 evaluations per prototype. A structured evaluation framework was developed to support consistent comparison across designs. Five criteria were used (Table 1):

a. Ease of use - intuitive grip, orientation, and facilitation of natural movement;
b. Safety - perceived thermal protection, grip stability, and secure handling;
c. Comfort - ergonomic fit, tactile qualities, and reduction of strain;
d. Robustness - perceived structural integrity and durability;
e. Proportions - visual balance and relationship to overall product form.

The selection of the five evaluation criteria was informed by contemporary research in ergonomics and hand-tool design, ensuring that the assessment captured both functional performance and perceptual user experience. Ease of use was included because intuitive grip configuration, wrist alignment, and clear affordances are central to efficient and error-free tool interaction, particularly in early-stage product evaluation. Safety was considered essential, as perceived grip stability, thermal protection, and slip resistance strongly influence user confidence and risk perception in handheld consumer products [17]. Comfort reflects tactile qualities, ergonomic fit, and pressure distribution, all of which have been shown to reduce muscular effort and perceived discomfort during tool use [18]. Robustness was included because users’ perceptions of structural integrity and durability directly affect trust, usability expectations, and long-term product acceptance, even prior to extended use (Figure 3). Finally, Proportions address the relationship between handle geometry, hand contact length, and visual balance, factors that influence both ergonomic comprehension and aesthetic judgement in user-centred design. Taken together, these criteria reflect current ergonomic thinking and provide a comprehensive framework for evaluating the functional, perceptual, and formal qualities of the prototypes in a peer-assessment context. Each criterion was rated using a 10-point Likert scale, where higher values indicated better performance. Definitions were provided to standardize interpretation among evaluators. The items were originally presented in Portuguese and were provided in bilingual form to ensure clarity for international students. Although the broader academic assignment included additional graded components (e.g., sketch development, construction quality, documentation), only the peer-evaluation scores were used in this research (Figure 4). Survey data were analysed using descriptive statistics, prototype ranking based on aggregated scores, and correlation analysis to explore relationships among the evaluated criteria.

Data analysis

The collected data were analysed using a combination of descriptive and multivariate methods. Mean scores were calculated for each criterion and prototype, enabling comparative ranking of design performance. Correlation analysis was used to examine relationships between evaluation criteria, while hierarchical clustering and principal component analysis (PCA) were applied to identify patterns and underlying dimensions within the dataset. MCDA was used to support the aggregation of criteria and the overall ranking of prototypes.

Analytical Methods

Evaluation dataset

Seventeen prototypes (P01–P17) were evaluated using the five criteria defined in the evaluation framework. The resulting dataset comprised a 17×5 matrix, with rows representing prototypes and columns corresponding to evaluation criteria.

Descriptive analysis and prototype ranking

Descriptive statistics were initially computed to provide an overview of the evaluation results and to characterise the performance of each prototype across the selected criteria. For each evaluation criterion, mean scores and standard deviations were calculated to assess central tendency and variability in student responses. In addition, score distributions were examined to identify patterns, dispersion, and potential outliers within the dataset. To enable comparison between prototypes, a total performance score was calculated for each design by summing the individual criterion scores, assuming equal weighting across all five criteria. This aggregated score provided a preliminary ranking of the prototypes, allowing the identification of higher- and lowerperforming designs. The descriptive analysis also facilitated an initial understanding of how consistently each prototype was evaluated and highlighted potential differences in student perceptions across the criteria.

Comparative and relational analysis

A one-way analysis of variance (ANOVA) was used to examine whether significant differences existed between evaluation criteria. In addition, Pearson correlation analysis was conducted to explore relationships between criteria and to identify how different aspects of design performance were perceived as interrelated. This method quantifies the strength and direction of linear associations between pairs of variables, producing correlation coefficients (r) ranging from −1 to +1. Positive values indicate that higher scores in one criterion are associated with higher scores in another, while negative values suggest an inverse relationship. Prior to analysis, assumptions of linearity and normality were considered to ensure the appropriateness of the Pearson method. The analysis enabled the identification of statistically significant relationships between criteria, providing insight into how students conceptually link different design attributes. For example, strong positive correlations may indicate that improvements in one aspect of design, such as proportions, are perceived to enhance related attributes such as comfort or ease of use. Overall, this analysis contributed to understanding potential dependencies and interactions between ergonomic, functional, and formal evaluation dimensions.

Pattern identification and dimensional analysis

To explore similarities between prototypes, hierarchical cluster analysis was applied to the standardised dataset, enabling the identification of groups of designs with comparable performance profiles. PCA was used to examine the underlying structure of the evaluation space and to identify dominant dimensions influencing design assessment. This facilitated the interpretation of relationships between criteria and the positioning of prototypes within a reduced-dimensional representation.

multi-criteria evaluation

MCDA approach was used to support the overall comparison of prototypes. Assuming equal weighting across the five criteria, a composite score was calculated for each design, allowing a systematic ranking based on combined performance. All analyses were conducted using Python-based data analysis tools./p>

Results

Perceived importance of evaluation criteria

The results (Table 2) show that students rated all five evaluation criteria highly, with scores clustering in the upper range of the scale. Safety received the highest importance rating (156 points, 21.8%). Ease of use (149 points, 20.8%) and comfort (148 points, 20.7%) received nearly identical ratings. Proportions (137 points, 19.1%) also obtained relatively high scores. Robustness received the lowest relative weighting (126 points, 17.6%), although it remained an important evaluation criterion overall.

t2

All five criteria received consistently high scores, indicating that participants considered usability, safety, comfort, robustness, and proportions as similarly relevant dimensions of handle design. Safety was rated as the most important criterion, followed closely by ease of use and comfort. Proportions also received high ratings, while robustness was slightly less emphasised. However, the relatively small variation across criteria suggests that students approached design evaluation in a balanced manner, without strongly privileging a single dimension.

Prototype evaluation and performance patterns

Students subsequently evaluated each prototype using the five criteria. The aggregated results of these evaluations are presented in (Table 3). The evaluation results reveal clear differences in performance across prototypes. A small group of designs (notably P01 and P16) achieved consistently high scores across all criteria, indicating a strong integration of ergonomic, functional, and formal qualities. In contrast, lower performing prototypes (e.g., P05, P15, and P17) showed consistently weaker evaluations across multiple criteria. Rather than excelling in a single dimension, the highest-ranked prototypes demonstrated balanced performance across criteria. This suggests that overall design quality was associated with the ability to integrate multiple attributes rather than optimise a single feature.

Distribution of evaluation scores

Descriptive statistical analysis was conducted to summarise the distribution of scores across the five evaluation criteria (Table 4). Robustness exhibited the highest mean score, while comfort showed the lowest mean value. Standard deviations indicate moderate variation in the student evaluations across prototypes. Descriptive statistics indicate moderate variation across prototypes, with robustness showing the highest average scores and comfort the lowest. However, differences between criteria were relatively small, reinforcing the observation that students evaluated designs using a multidimensional perspective.

Comparison between evaluation criteria

A one-way ANOVA was conducted to determine whether significant differences existed between the mean scores of the five evaluation criteria. The analysis indicated that the differences between criteria were not statistically significant (F = 1.45, p = 0.226). The analysis did not reveal statistically significant differences between criteria, suggesting that no single dimension dominated the evaluation process. This further supports the interpretation that students applied the evaluation framework in a consistent and balanced manner across all criteria.

Relationships between evaluation criteria

Pearson correlation analysis was performed to examine relationships between the evaluation criteria (Table 5). Correlation analysis revealed strong positive relationships between ease of use, comfort, and proportions. This indicates that these attributes were perceived as closely interconnected, suggesting that geometric coherence and ergonomic fit contribute simultaneously to usability and comfort. A strong association was also observed between safety and robustness, indicating that perceptions of structural integrity are closely linked to perceived user security.

Prototype ranking

MCDA using equal weighting for the five evaluation criteria was conducted to determine overall prototype performance (Table 6). The ranking results confirm that the highest-performing prototypes combined consistent performance across all criteria. Designs such as P01 and P16 achieved high overall scores by maintaining balanced evaluations rather than excelling in isolated aspects. Lower-ranked prototypes tended to underperform across multiple criteria, reinforcing the importance of integrated design solutions.

Grouping of design outcomes

Hierarchical cluster analysis grouped the prototypes into three performance categories (Table 7). Cluster analysis identified three distinct groups of prototypes corresponding to high- , medium-, and low-performing designs. This grouping aligns with the ranking results and highlights clear differences in how design solutions combine ergonomic and formal attributes. Highperforming prototypes are characterised by coherence across criteria, while lower-performing designs exhibit inconsistencies that negatively affect overall evaluation.

Discussion

This study provides insight into how ergonomic and formal qualities are interpreted and evaluated within a studio-based design context. The findings indicate that participants approached evaluation in a consistently multidimensional manner, attributing comparable importance to usability, safety, comfort, robustness, and proportions. Rather than privileging a single aspect of performance, the results suggest that design quality is understood as an integration of multiple, interrelated attributes [20,21]. The relatively balanced weighting of criteria is particularly significant in relation to design practice. It indicates that even at early stages of training, designers are able to consider ergonomic, functional, and formal aspects simultaneously. This reflects the integrative nature of design thinking, where usability, safety, and aesthetic coherence are not treated as separate domains but as mutually reinforcing dimensions of product experience [22,23].

Among the evaluated criteria, safety emerged as the most prominent, highlighting the importance of perceived stability, grip security, and user protection in shaping design evaluation. Ease of use and comfort were closely aligned, reinforcing their conceptual interdependence and their role in shaping intuitive interaction [24]. The strong emphasis on proportions further suggests that visual coherence and geometric balance are not only aesthetic concerns but are closely linked to perceived usability and ergonomic fit [25]. By contrast, robustness was consistently rated slightly lower, which may indicate the difficulty of assessing structural performance through visual and tactile inspection alone. This points to a limitation inherent in early-stage design evaluation, where perceptual qualities are more readily assessed than underlying technical properties [26,27]. It also highlights the importance of integrating material and structural considerations more explicitly into design processes.hould focus on exercises that provide challenges to seal stability.

The analysis of prototype performance further reinforces the importance of integration across criteria. High-performing designs did not excel in a single dimension but demonstrated consistency across multiple attributes, suggesting that overall design quality emerges from coherence rather than isolated optimisation [28]. Conversely, lower-performing prototypes tended to exhibit weaknesses across several criteria, indicating that deficiencies in one aspect can influence overall perception. This pattern is consistent with evaluative biases such as the halo effect, where impressions in one-dimension affect judgements in others [29]. The relationships identified between evaluation criteria provide additional insight into how design attributes are conceptually organised. Strong associations between proportions, ease of use, and comfort suggest that geometric configuration plays a central role in shaping both ergonomic fit and perceived usability [30]. Similarly, the link between safety and robustness indicates that perceptions of structural integrity contribute directly to feelings of user security.

The identification of broader groupings of prototypes further supports this interpretation, suggesting that design outcomes can be understood in terms of how effectively different attributes are combined, rather than as independent performance measures. In this sense, the evaluation process reveals implicit criteria of coherence and balance that underpin judgements of design quality. A key contribution of this study lies in demonstrating how data-informed approaches can support design evaluation within studio practice. By combining peer-based assessment with structured analytical methods, the framework makes patterns of evaluation visible and enables more systematic comparison between design alternatives. Rather than replacing qualitative judgement, this approach complements it by providing a structured means of articulating and comparing design decisions [31,32]. This integration of qualitative and quantitative evaluation is particularly relevant in design contexts where judgement is often tacit and difficult to externalise. Making evaluation criteria explicit and measurable allows designers to reflect more critically on their decisions and to understand how different attributes contribute to overall performance [33-37].

Conclusions

This study examined how ergonomic and formal qualities are evaluated within a studio-based design context through a structured, data-informed framework. The findings show that design evaluation is inherently multidimensional, with usability, safety, comfort, robustness, and proportions considered as interrelated aspects of overall performance rather than independent criteria. The results highlight the central role of usability-related attributes, particularly safety, ease of use, and comfort, alongside the importance of geometric coherence in shaping perceived design quality. The comparatively lower emphasis on robustness suggests the difficulty of assessing structural performance through perceptual evaluation alone, pointing to a limitation of early-stage design assessment. A key contribution of this work lies in demonstrating how datainformed approaches can support the evaluation of design alternatives. By combining peer-based assessment with structured analytical methods, the proposed framework enables systematic comparison, reveals relationships between design attributes, and makes evaluative patterns more explicit. This supports a more transparent and reflective approach to design decisionmaking within studio practice. The findings also suggest that high-performing designs are characterised by consistency across multiple attributes, reinforcing the importance of integration and coherence in design outcomes. Conversely, weaker designs tend to exhibit imbalances across criteria, indicating that deficiencies in one aspect can influence overall perception.

Several limitations should be acknowledged. The study is based on a single cohort and relies on peer evaluation within a controlled setting, which may limit generalisability. In addition, the absence of objective ergonomic measurements restricts the assessment of actual performance beyond perceived qualities. Future work could extend this approach by integrating user testing, biomechanical analysis, and expert evaluation to complement subjective assessment. Further research may also explore how data-informed evaluation frameworks can be applied across different design contexts and levels of expertise. Overall, this study demonstrates that structured and data-informed evaluation can enhance the way design alternatives are compared and interpreted. By linking ergonomic considerations with analytical methods, the framework contributes to a more systematic and reflective approach to design evaluation in studio practice.

Declaration of Competing Interests

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Funding

This work is funded by national funds through FCTFoundation for Science and Technology, I.P., under the Project UID/05237/2025 (https://doi.org/10.54499/UID/05237/2025).

References

1. Tichauer ER, Gage H (1977) Ergonomic principles basic to hand tool design. American Industrial Hygiene Association Journal 38(11): 622-634.
2. Cacha CA (1999) Ergonomics and safety in hand tool design. CRC Press.
3. Veisi H, Choobineh A, Ghaem H, Shafiee Z (2019) The effect of hand tools handle shape on upper extremity comfort and postural discomfort among hand-woven shoemaking workers. International Journal of Industrial Ergonomics 74: 102833.
4. Dianat I, Nedaei M, Nezami, MAM (2015) The effects of tool handle shape on hand performance, usability and discomfort using masons trowels. International Journal of Industrial Ergonomics 45: 13-20.
5. Pheasant S, Haslegrave CM (2018) Bodyspace: Anthropometry, ergonomics and the design of work. CRC press.
6. Rastgoo Balov L, Allahyari T, Jafari M, Hajaghazadeh M (2026) Hand anthropometry grip strength and manual dexterity: A correlational study among dental students. Sage Open Medicine 14: 20503121261428206.
7. Quattrocchi A, Garufi G, Gugliandolo G, De Marchis C, Collufio D, et al., (2024) Handgrip strength in health applications: a review of the measurement methodologies and influencing factors. Sensors 24(16): 5100.
8. Maleki Ghahfarokhi A, et al. (2023) Handle geometry and usability. Applied Ergonomics 110: 104001.
9. Alam MD, Khan IA (2025) Relation between grip force, ergonomic interventions, and task performance: a review. International Journal on Interactive Design and Manufacturing (IJIDeM) 19(7): 4683-4713.
10. Dul J, Neumann WP (2009) Ergonomics contributions to company strategies. Applied ergonomics 40(4): 745-752.
11. Antony J (2014) Readiness factors for the Lean Six Sigma journey in the higher education sector. International Journal of Productivity and Performance Management 63(2): 257-264.
12. Agung ICS, Siswantoro T, Bawono B, Anggoro PW (2023) Ergonomic cup handle design analysis at Naruna Ceramic Studio: A literature review. Cogent Engineering 10(1): 2253035.
13. Jain R, Rana KB, Meena ML, Sidh S (2021) Ergonomic assessment and hand tool redesign for the small scale furniture industry. Materials Today: Proceedings 44: 4952-4955.
14. Waller S, Bradley M, Hosking I, Clarkson PJ (2015) Making the case for inclusive design. Applied ergonomics 46: 297-303.
15. Clarkson PJ, Coleman R, Keates S, Lebbon C (2013) Inclusive design: Design for the whole population.
16. Wollesen B, Gräf J, Schumacher N, Meyer G, Wanstrath M, et al., (2020) Influences of neck and/or wrist pain on hand grip strength of industrial quality proofing workers. Safety and Health at Work 11(4): 458-465.
17. Stanton NA, Salmon PM, Rafferty LA, Walker GH, Baber C, et al., (2017) Human factors methods: a practical guide for engineering and design. CRC Press 656.
18. Kong YK, Lowe BD (2005) Evaluation of handle diameters and orientations in a maximum torque task. International Journal of Industrial Ergonomics 35(12): 1073-1084.
19. Seo NJ, Armstrong TJ, Chaffin DB, Ashton-Miller JA (2008) The effect of handle friction and inward or outward torque on maximum axial push force. Human Factors 50(2): 227-236.
20. Crilly N, Moultrie J, Clarkson PJ (2004) Seeing things: consumer response to the visual domain in product design. Design studies, 25(6): 547-577.
21. Desmet P, Hekkert P (2007) Framework of product experience. International journal of design 1(1): 57-66.
22. Dorst K (2011) The core of ‘design thinking’and its application. Design studies 32(6): 521-532.
23. Norman Donald A (2013) The design of everyday things. MIT Press.
24. ISO (2019) 9241-210: 2019 Ergonomics of human-system interaction–Part 210: human-centred design for interactive systems. Geneva, Switzerland: International Organization for Standardization.
25. Lidwell W, Holden K, Butler J (2010) Universal principles of design, revised and updated: 125 ways to enhance usability, influence perception increase appeal make better design decisions and teach through design. Rockport Pub.
26. Ahmed S, Wallace KM, Blessing LT (2003) Understanding the differences between how novice and experienced designers approach design tasks. Research in engineering design 14(1): 1-11.
27. Ulrich KT, Eppinger SD, Yang MC (2019) Product design and development. Boston: McGraw-Hill Education.
28. Hekkert P (2006) Design aesthetics: principles of pleasure in design. Psychology science 48(2): 157.
29. Nisbett RE, Wilson TD (1977) The halo effect: Evidence for unconscious alteration of judgments. Journal of personality and social psychology 35(4): 250-256.
30. Harih G, Dolšak B (2013) Tool-handle design based on a digital human hand model. International journal of industrial ergonomics 43(4): 288-295.
31. Cross N (2023) Design thinking: Understanding how designers think and work. Bloomsbury Publishing.
32. Schön DA (2017) The reflective practitioner: How professionals think in action. Routledge 384
33. Ajjawi R, Tai J, Dawson P, Boud D (2018) Conceptualising evaluative judgement for sustainable assessment in higher education. In Developing evaluative judgement in higher education. Routledge 11.
34. Carless D, Boud D (2018) The development of student feedback literacy: Enabling uptake of feedback. Assessment & evaluation in higher education 43(8): 1315-1325.
35. Gray CM (2013) Informal peer critique and the negotiation of habitus in a design studio. Art, Design & Communication in Higher Education 12(2): 195-209.
36. Nicol D, Thomson A, Breslin C (2014a) Rethinking feedback practices in higher education: a peer review perspective. Assessment & evaluation in higher education 39(1): 102-122.
37. Yilmaz S, Daly SR (2016) Feedback in concept development: Comparing design disciplines. Design Studies 45: 137-158.